Compare commits

..
9 Commits
Author SHA1 Message Date
infosave2007 4fa1b8ea22 feat(settings): pick a local model from a dropdown of existing .cmf files
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
- New GET /admin/api/cmf/files lists the .cmf files (and tensor-dir .cmf) under
  models_dir, with sizes.
- Settings → Local models: the per-row model field is now a dropdown of those
  files instead of a hand-typed path, and auto-fills the id from the filename.
2026-07-09 15:25:30 +03:00
infosave2007 357441e0da chore(release): v0.2.8 — multiple managed local models
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
- The gateway can now run several local .cmf models at once: a [[cmf.servers]]
  list, one supervised `cortiq serve` per entry on its own port, each registered
  in the pool and independently routable/pinnable. The single legacy
  local_model/local_port still works (back-compat via effective_servers()).
- Settings → Local models is now a rows editor (id · path · port + check · threads
  · GPU) with add/remove and per-server serving status.
- Deleting a managed model stops just its server and removes its file — unless
  another model still points at the same file (shared-file guard).
2026-07-09 12:00:10 +03:00
infosave2007 9d2181e82b chore(release): v0.2.7 — honest pins, visible managed model, physical delete
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
- A pinned playground model that is unavailable now errors clearly instead of
  silently answering from the managed local model (route_source no longer lies).
- The managed CMF model is surfaced in GET /admin/api/models (marked "managed",
  shown read-only in the UI) so the real running model is visible and pinnable
  without a hand-maintained [[models]] entry; a fresh install still lists nothing.
- Deleting a model now removes its local .cmf physically (file or tensor-dir);
  deleting the managed model also stops the server and clears the [cmf] config.
  Routing is stripped of the deleted id so the config still validates.
2026-07-08 21:14:03 +03:00
infosave2007 30378f7b6a chore: fix clippy lints to green the lint CI
- import.rs / promotion.rs: sort_by(|a,b| b.k.cmp(&a.k)) -> sort_by_key(Reverse(k))
- pipeline.rs: skip_while(!p).next() -> find(p)
Behaviour-preserving; completes the CI fix started by the cargo fmt commit.
2026-07-08 20:09:54 +03:00
infosave2007 b4f78bd192 chore: cargo fmt — fix lint 'Format' CI job (red since v0.2.3)
Whole-repo rustfmt drift; behaviour unchanged (whitespace/layout only).
All test platforms already passed; this greens the lint job.
2026-07-08 19:32:35 +03:00
infosave2007 3c90dad9b9 chore(release): v0.2.6 — Settings UX, empty pool by default, CMF port check
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
- Settings: "?" help tooltips on every option; router key auto-probes the
  connection right after saving (no more hanging "unavailable")
- Local models card reworked into 3 clear steps (install cortiq → add model →
  run locally); removed the hand-entered .cmf path / pool-id fields — models
  are added via the Import tab and the managed server registers them
- Fresh install ships an EMPTY model pool; the local .cmf is auto-registered by
  the managed CMF server, so it needs no static [[models]] entry
- New: port-availability check — GET /admin/api/cmf/port + "Check port" button
  with next-free-port suggestion. bind()+connect() probe so Docker-proxied /
  dual-stack ports (e.g. 8081) are correctly detected as occupied
- /admin/api/cmf now reports the serving model + port; card shows "▶ serving …"
- On-demand runtime install: POST /admin/api/cmf/install
2026-07-08 19:07:24 +03:00
infosave2007 25f93d1a27 chore: ship the full working frontend + backend
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
The crates.io/release build shipped a STALE frontend (committed web/js was behind
the working tree), so the admin console hung on load. Commit the current working
web/js (app, api, dashboard, login, import view, i18n de/es/fr/tr/zh, css) and the
backend WIP (import.rs, admin/assets.rs) so the released console matches the dev
build. Verified: clean `cargo build --release`, empty-pool start, working
dashboard served.
2026-07-08 17:58:31 +03:00
infosave2007 0f89cdaa07 chore(release): v0.2.4
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-07-08 17:39:43 +03:00
infosave2007 d211eb6150 feat: fresh install starts with an empty model pool
- config.rs: validate() no longer errors on zero models — an empty pool is
  allowed (warns instead), so the admin console comes up on a fresh install and
  the user adds models in /admin -> Models (or points [cmf] at a local .cmf).
- gateway.example.toml: ships an EMPTY pool (no seed cmf-local/local-embed, empty
  routing), so `cp example -> gateway.toml && run` shows an empty list — what a
  new user expects.
2026-07-08 17:39:43 +03:00
27 changed files with 1730 additions and 293 deletions
Generated
+1 -1
View File
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "cortiq-gateway"
version = "0.2.3"
version = "0.2.9"
dependencies = [
"anyhow",
"arc-swap",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "cortiq-gateway"
version = "0.2.3"
version = "0.2.9"
edition = "2021"
rust-version = "1.75"
license = "Apache-2.0"
+18 -29
View File
@@ -28,40 +28,29 @@ cache_ttl = "60s"
profile = "balanced" # cost-saver | balanced | quality-first
# ============================================================================
# Model pool. This example ships ONE local CMF model, served by the managed
# cortiq server (see the [cmf] section below). Add more models — local or cloud —
# by copying the [[models]] shape: provider = "openai"/"anthropic", and for a
# hosted provider an api_key_env = "..." (never put secrets in this file).
# Model pool — EMPTY by default. A fresh gateway starts with no models and an
# empty pool; add them in the admin console (/admin → Models), or run a local
# .cmf via the [cmf] section below. To add one by hand, copy this shape:
#
# [[models]]
# id = "my-model"
# provider = "openai" # openai | anthropic (openai also covers
# base_url = "http://127.0.0.1:8081/v1" # OpenRouter/Together/Groq/vLLM/Ollama)
# model = "my-model"
# # api_key_env = "OPENAI_API_KEY" # for hosted providers (never inline a key)
# cost_tier = "local"
# caps = ["tools"]
# ============================================================================
[[models]]
id = "cmf-local"
provider = "openai"
base_url = "http://127.0.0.1:8081/v1" # the managed cortiq server (see [cmf])
model = "cmf-local"
cost_tier = "local"
price_in = 0.0
price_out = 0.0
caps = ["tools"]
# --- embedding model (optional: /v1/embeddings + the semantic cache) ---------
[[models]]
id = "local-embed"
provider = "openai"
base_url = "http://localhost:8000/v1" # a local embeddings server (vLLM / llama.cpp / Ollama)
model = "nomic-embed-text"
cost_tier = "local"
kind = "embedding"
# ============================================================================
# Routing table: complexity tier → ordered pool (priority = fallback chain).
# Add cloud models above and list them here for cost/quality-based fallback.
# Leave default empty for a fresh install; set it to a model id once you add one.
# ============================================================================
[routing]
low = ["cmf-local"]
medium = ["cmf-local"]
high = ["cmf-local"]
default = "cmf-local" # used when the router is unreachable
low = []
medium = []
high = []
default = "" # set to a model id once you add one
[routing.policy]
mode = "fixed_table" # fixed_table | cost_aware
@@ -153,7 +142,7 @@ max_entries = 1000
manage_server = false # true → gateway spawns `cortiq serve` at startup
local_model = "models/model.cmf" # path to the .cmf to serve
local_host = "127.0.0.1"
local_port = 8081 # must match the cmf-local base_url above
local_port = 8081 # confirm it's free in Settings → "Check port" (8081 often clashes with Docker/Apache)
model_id = "cmf-local" # id it registers under (matches [[models]])
threads = 8 # decode worker threads (CMF_THREADS); ~3x vs serial
gpu = false # offload matvecs to GPU (Metal/wgpu); mainly a prefill win
+12 -2
View File
@@ -16,11 +16,18 @@ use rust_embed::RustEmbed;
struct WebAssets;
fn serve_file(path: &str) -> Response {
// The panel is rebuilt often (assets are embedded at compile time and served
// from memory), so tell the browser to always revalidate — otherwise a stale
// cached SPA hides freshly shipped fixes until a hard reload.
const NO_CACHE: &str = "no-store, must-revalidate";
match WebAssets::get(path) {
Some(content) => {
let mime = mime_guess::from_path(path).first_or_octet_stream();
(
[(header::CONTENT_TYPE, mime.as_ref())],
[
(header::CONTENT_TYPE, mime.as_ref()),
(header::CACHE_CONTROL, NO_CACHE),
],
content.data.into_owned(),
)
.into_response()
@@ -29,7 +36,10 @@ fn serve_file(path: &str) -> Response {
// SPA: unknown path under /admin → serve index.html
match WebAssets::get("index.html") {
Some(content) => (
[(header::CONTENT_TYPE, "text/html; charset=utf-8")],
[
(header::CONTENT_TYPE, "text/html; charset=utf-8"),
(header::CACHE_CONTROL, NO_CACHE),
],
content.data.into_owned(),
)
.into_response(),
+283 -19
View File
@@ -26,6 +26,7 @@ use axum::{
use serde::Deserialize;
use serde_json::{json, Value};
use std::collections::HashMap;
use std::net::ToSocketAddrs;
use std::time::Instant;
// ───────────────────────────── common types / helpers ───────────────────────
@@ -168,9 +169,16 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
.route("/admin/api/shadow", get(get_shadow))
.route("/admin/api/hf/search", get(hf_search))
.route("/admin/api/import", get(list_imports).post(start_import))
.route("/admin/api/import/:job", get(import_status).delete(delete_import))
.route(
"/admin/api/import/:job",
get(import_status).delete(delete_import),
)
.route("/admin/api/import/:job/cancel", post(cancel_import))
.route("/admin/api/import/:job/register", post(register_import))
.route("/admin/api/cmf", get(cmf_status))
.route("/admin/api/cmf/install", post(cmf_install))
.route("/admin/api/cmf/port", get(cmf_port_check))
.route("/admin/api/cmf/files", get(cmf_files))
.route("/admin/api/requests", get(get_requests))
.route("/admin/api/test", post(run_test))
.route("/admin/api/test/stream", post(run_test_stream))
@@ -419,7 +427,7 @@ async fn put_config(State(state): State<SharedState>, Json(cfg): Json<Config>) -
async fn list_models(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let models: Vec<Value> = live
let mut models: Vec<Value> = live
.cfg
.models
.iter()
@@ -433,6 +441,31 @@ async fn list_models(State(state): State<SharedState>) -> ApiResult {
v
})
.collect();
// Managed local CMF servers are registered in the routing registry from the
// [cmf] config, not from [[models]]. Surface each here (marked managed, so
// the UI shows it read-only) so the real running models are visible and
// pinnable without a hand-maintained static pool entry. A fresh install with
// no managed model still returns an empty list.
let cmf = &live.cfg.cmf;
for s in cmf.effective_servers() {
if models.iter().any(|m| m["id"] == json!(s.id)) {
continue;
}
models.push(json!({
"id": s.id,
"provider": "openai",
"base_url": format!("http://{}:{}/v1", cmf.local_host, s.port),
"model": "cortiq",
"cost_tier": "local",
"price_in": 0.0,
"price_out": 0.0,
"kind": "chat",
"caps": [],
"key_source": "none",
"managed": true,
}));
}
ok(json!({ "models": models }))
}
@@ -466,15 +499,109 @@ async fn update_model(
ok(json!({ "ok": true }))
}
/// Remove a `.cmf` path — it may be a plain file or a tensor-dir. Returns whether
/// something was actually removed.
fn remove_cmf_path(path: &std::path::Path) -> bool {
if path.is_dir() {
std::fs::remove_dir_all(path).is_ok()
} else if path.is_file() {
std::fs::remove_file(path).is_ok()
} else {
false
}
}
/// The local `.cmf` backing a pool model, if it is a local CMF model whose file
/// lives under `models_dir`. Cloud models (real remote base_url) return None.
/// Registered imports store `model` = the file stem under models_dir.
fn cmf_file_for_model(
cmf: &crate::config::CmfCfg,
m: &crate::config::ModelCfg,
) -> Option<std::path::PathBuf> {
let local = m.base_url == cmf.cortiq_server_url
|| m.base_url
.contains(&format!("{}:{}", cmf.local_host, cmf.local_port));
let stem = m.model.trim();
if !local || stem.is_empty() || stem.contains('/') || stem.contains("..") {
return None; // not local, or an unsafe/underivable path
}
Some(std::path::Path::new(&cmf.models_dir).join(format!("{stem}.cmf")))
}
/// Drop a model id from routing (default + every tier) so `validate()` still
/// passes after the model is gone.
fn strip_id_from_routing(routing: &mut RoutingCfg, id: &str) {
if routing.default == id {
routing.default = String::new();
}
for tt in routing.tiers.values_mut() {
match tt {
crate::config::TierTargets::List(v) => v.retain(|x| x != id),
crate::config::TierTargets::One(s) if s == id => {
*tt = crate::config::TierTargets::List(Vec::new());
}
crate::config::TierTargets::One(_) => {}
}
}
}
/// Whether any managed server or static model in `cfg` still uses `path` as its
/// local `.cmf`. Guards against deleting a file that another model shares.
fn cmf_path_referenced(cfg: &crate::config::Config, path: &std::path::Path) -> bool {
cfg.cmf
.effective_servers()
.iter()
.any(|s| std::path::Path::new(&s.model) == path)
|| cfg
.models
.iter()
.any(|m| cmf_file_for_model(&cfg.cmf, m).as_deref() == Some(path))
}
async fn delete_model(State(state): State<SharedState>, Path(id): Path<String>) -> ApiResult {
let mut cfg = current_cfg(&state);
// Managed local models live in [cmf], not [[models]]. Deleting one stops its
// server, removes it from the managed config, and deletes its file — unless
// another model still points at the same file.
let managed = cfg.cmf.effective_servers().into_iter().find(|s| s.id == id);
if let Some(server) = managed {
if !cfg.models.iter().any(|x| x.id == id) {
if cfg.cmf.servers.iter().any(|s| s.id == id) {
cfg.cmf.servers.retain(|s| s.id != id); // multi-model list entry
} else {
cfg.cmf.local_model = String::new(); // legacy single model
cfg.cmf.manage_server = false;
}
strip_id_from_routing(&mut cfg.routing, &id);
let path = std::path::PathBuf::from(&server.model);
let shared = cmf_path_referenced(&cfg, &path);
state.cmf.stop_one(&id).await;
state.reload(cfg)?;
let file_removed = if shared {
false
} else {
remove_cmf_path(&path)
};
return ok(json!({ "ok": true, "file_removed": file_removed }));
}
}
let victim = cfg.models.iter().find(|x| x.id == id).cloned();
let before = cfg.models.len();
cfg.models.retain(|x| x.id != id);
if cfg.models.len() == before {
return Err(ApiError::not_found(format!("model '{id}' not found")));
}
strip_id_from_routing(&mut cfg.routing, &id);
let file = victim
.as_ref()
.and_then(|m| cmf_file_for_model(&cfg.cmf, m));
// Only remove the file if no remaining model still references it.
let file = file.filter(|p| !cmf_path_referenced(&cfg, p));
state.reload(cfg)?;
ok(json!({ "ok": true }))
let file_removed = file.map(|p| remove_cmf_path(&p)).unwrap_or(false);
ok(json!({ "ok": true, "file_removed": file_removed }))
}
async fn probe_model(State(state): State<SharedState>, Path(id): Path<String>) -> ApiResult {
@@ -662,6 +789,7 @@ async fn put_settings(State(state): State<SharedState>, Json(b): Json<SettingsBo
|| v.local_model != cfg.cmf.local_model
|| v.threads != cfg.cmf.threads
|| v.gpu != cfg.cmf.gpu
|| v.servers != cfg.cmf.servers
{
needs_restart = true;
}
@@ -671,6 +799,154 @@ async fn put_settings(State(state): State<SharedState>, Json(b): Json<SettingsBo
ok(json!({ "ok": true, "needs_restart": needs_restart }))
}
// ───────────────────────────── cmf runtime ─────────────────────────────────
/// Status of the `cortiq` CLI (the CMF format runtime): is it installed, which
/// version, and whether a managed server is running. Drives the "Install cortiq"
/// button + status in the Settings → Local models card.
async fn cmf_status(State(state): State<SharedState>) -> ApiResult {
let cmf = state.live().cfg.cmf.clone();
let ver = crate::cmf_runtime::installed_version(&cmf.cortiq_bin);
let st = state.cmf.status();
// Each configured managed server, merged with its live runtime status.
let servers: Vec<Value> = cmf
.effective_servers()
.iter()
.map(|s| {
let live = st.servers.iter().find(|x| x.id == s.id);
json!({
"id": s.id,
"model": s.model,
"port": s.port,
"threads": s.threads,
"gpu": s.gpu,
"running": live.map(|l| l.running).unwrap_or(false),
"healthy": live.map(|l| l.healthy).unwrap_or(false),
"last_error": live.and_then(|l| l.last_error.clone()),
})
})
.collect();
ok(json!({
"installed": ver.is_some(),
"version": ver,
"latest": st.latest_version,
"manage_server": cmf.manage_server,
"local_host": cmf.local_host,
"servers": servers,
"log": st.log,
}))
}
/// Install/upgrade the `cortiq` CLI from crates.io on demand (spawned in the
/// background; poll `GET /admin/api/cmf` for `installed`/`version`).
async fn cmf_install(State(state): State<SharedState>) -> ApiResult {
let bin = state.live().cfg.cmf.cortiq_bin.clone();
tokio::spawn(crate::cmf_runtime::install_now(state.cmf.clone(), bin));
ok(json!({ "started": true }))
}
/// List the `.cmf` model files available under `models_dir` so the UI can offer
/// them as a dropdown instead of a hand-typed path. Includes tensor-dir `.cmf`.
async fn cmf_files(State(state): State<SharedState>) -> ApiResult {
let dir = state.live().cfg.cmf.models_dir.clone();
let mut entries: Vec<(String, String, Option<u64>, bool)> = Vec::new();
if let Ok(rd) = std::fs::read_dir(&dir) {
for e in rd.flatten() {
let name = e.file_name().to_string_lossy().to_string();
if !name.ends_with(".cmf") {
continue;
}
let p = e.path();
let is_dir = p.is_dir();
let size = if is_dir {
None
} else {
std::fs::metadata(&p).ok().map(|m| m.len())
};
let path = format!("{}/{}", dir.trim_end_matches('/'), name);
entries.push((name, path, size, is_dir));
}
}
entries.sort_by(|a, b| a.0.cmp(&b.0));
let files: Vec<Value> = entries
.into_iter()
.map(|(name, path, size, is_dir)| json!({ "name": name, "path": path, "size": size, "is_dir": is_dir }))
.collect();
ok(json!({ "dir": dir, "files": files }))
}
#[derive(Deserialize)]
struct PortQuery {
port: u16,
}
/// Check whether a TCP port is free to bind on the CMF local host, so a user can
/// pick an available port before enabling the managed server (avoids clashing
/// with something already on the default 8081). If the port is the one this
/// gateway's own managed server already runs on, that is reported distinctly.
/// When occupied, `suggested` holds the next free port so the UI can offer it.
async fn cmf_port_check(State(state): State<SharedState>, Query(q): Query<PortQuery>) -> ApiResult {
let cmf = state.live().cfg.cmf.clone();
let host = if cmf.local_host.trim().is_empty() {
"127.0.0.1".to_string()
} else {
cmf.local_host.clone()
};
let st = state.cmf.status();
let port = q.port;
let running_here = st.servers.iter().any(|s| s.port == port && s.running);
// The bind+connect probe is blocking std::net; run it off the async runtime.
let (available, detail, suggested) = tokio::task::spawn_blocking(move || {
if running_here {
return (false, "in use by this gateway's managed CMF server", None);
}
if port_is_free(&host, port) {
(true, "free", None)
} else {
(
false,
"occupied by another process",
next_free_port(&host, port),
)
}
})
.await
.unwrap_or((false, "port check failed", None));
ok(json!({
"port": port,
"available": available,
"detail": detail,
"suggested": suggested,
}))
}
/// Whether `port` is truly free on `host`. A bind test alone is not enough: a
/// Docker-published / dual-stack listener can leave an IPv4 bind succeeding even
/// though connections to the port are forwarded to a container. So a port counts
/// as free only if it is BOTH bindable AND nothing answers a connect attempt.
fn port_is_free(host: &str, port: u16) -> bool {
if std::net::TcpListener::bind((host, port)).is_err() {
return false; // something is bound to exactly this address
}
// catch proxied/forwarded ports a bind test misses (e.g. Docker on macOS)
let reachable = (host, port)
.to_socket_addrs()
.ok()
.and_then(|mut it| it.next())
.map(|addr| {
std::net::TcpStream::connect_timeout(&addr, std::time::Duration::from_millis(200))
.is_ok()
})
.unwrap_or(false);
!reachable
}
/// First free port in (from, from+50], or None if all are taken.
fn next_free_port(host: &str, from: u16) -> Option<u16> {
(from.saturating_add(1)..=from.saturating_add(50)).find(|p| port_is_free(host, *p))
}
// ───────────────────────────── keys ────────────────────────────────────────
async fn list_keys(State(state): State<SharedState>) -> ApiResult {
@@ -869,10 +1145,7 @@ async fn start_import(
ok(json!({ "job": id }))
}
async fn import_status(
State(state): State<SharedState>,
Path(job): Path<String>,
) -> ApiResult {
async fn import_status(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
match state.imports.get(&job) {
Some(j) => ok(serde_json::to_value(j).unwrap_or_else(|_| json!({}))),
None => Err(ApiError::not_found(format!("job '{job}' not found"))),
@@ -880,10 +1153,7 @@ async fn import_status(
}
/// Cancel a running conversion: kills the converter and removes partial output.
async fn cancel_import(
State(state): State<SharedState>,
Path(job): Path<String>,
) -> ApiResult {
async fn cancel_import(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
if state.imports.get(&job).is_none() {
return Err(ApiError::not_found(format!("job '{job}' not found")));
}
@@ -892,10 +1162,7 @@ async fn cancel_import(
}
/// Delete a finished conversion and its converted `.cmf` file(s) from disk.
async fn delete_import(
State(state): State<SharedState>,
Path(job): Path<String>,
) -> ApiResult {
async fn delete_import(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
if state.imports.get(&job).is_none() {
return Err(ApiError::not_found(format!("job '{job}' not found")));
}
@@ -904,10 +1171,7 @@ async fn delete_import(
}
/// Register a finished `.cmf` as a local model (OpenAI provider → cortiq-server).
async fn register_import(
State(state): State<SharedState>,
Path(job): Path<String>,
) -> ApiResult {
async fn register_import(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
let j = state
.imports
.get(&job)
+220 -84
View File
@@ -1,51 +1,68 @@
//! Local CMF model server lifecycle.
//!
//! Closes the loop with <https://github.com/infosave2007/cmf>: install the
//! `cortiq` CLI from crates.io, keep it up to date, and run `cortiq serve` as a
//! managed child process. The served model exposes an OpenAI-compatible API, so
//! the registry registers it as an ordinary provider and the gateway routes to
//! it like any other backend.
//! `cortiq` CLI from crates.io, keep it up to date, and run one `cortiq serve`
//! per configured local model as a managed child process. Each served model
//! exposes an OpenAI-compatible API, so the registry registers it as an ordinary
//! provider and the gateway routes to it like any other backend.
use crate::config::CmfCfg;
use crate::config::{CmfCfg, CmfServer};
use std::process::Stdio;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use tokio::process::{Child, Command};
/// Observable state of the managed local CMF server (surfaced to the admin API).
/// Observable state of one managed local model server.
#[derive(Clone, Debug, Default, serde::Serialize)]
pub struct CmfServerStatus {
/// Pool id it is registered under.
pub id: String,
/// Path of the `.cmf` being served.
pub model: String,
/// Port the server binds to.
pub port: u16,
/// OpenAI base URL the local server is registered under.
pub base_url: String,
/// The child `cortiq serve` process has been spawned.
pub running: bool,
/// `/healthz` on the local server answered 200.
pub healthy: bool,
/// Last fatal error for this server, if any.
pub last_error: Option<String>,
}
/// Observable state of the managed local CMF layer (surfaced to the admin API).
#[derive(Clone, Debug, Default, serde::Serialize)]
pub struct CmfStatus {
/// `manage_server` is on and a `local_model` is configured.
/// `manage_server` is on and at least one local model is configured.
pub enabled: bool,
/// Version reported by `cortiq --version` (None = not installed).
pub installed_version: Option<String>,
/// Newest `cortiq-cli` on crates.io (populated when `auto_update` runs).
pub latest_version: Option<String>,
/// The child `cortiq serve` process has been spawned.
pub running: bool,
/// `/healthz` on the local server answered 200.
pub healthy: bool,
/// OpenAI base URL the local server is registered under.
pub base_url: String,
/// Path of the `.cmf` being served.
pub model: String,
/// Last fatal error, if any.
pub last_error: Option<String>,
/// Per-model server status.
pub servers: Vec<CmfServerStatus>,
/// Recent lifecycle log lines (bounded).
pub log: Vec<String>,
}
/// Holds the managed child process and its live status.
/// A spawned server: its pool id and the child process handle.
struct Managed {
id: String,
child: Child,
}
/// Holds the managed child processes and their live status.
pub struct CmfRuntime {
status: Mutex<CmfStatus>,
child: Mutex<Option<Child>>,
children: Mutex<Vec<Managed>>,
}
impl CmfRuntime {
pub fn new() -> Arc<Self> {
Arc::new(Self {
status: Mutex::new(CmfStatus::default()),
child: Mutex::new(None),
children: Mutex::new(Vec::new()),
})
}
@@ -59,32 +76,115 @@ impl CmfRuntime {
let mut s = self.status.lock().unwrap();
s.log.push(msg);
let n = s.log.len();
if n > 50 {
s.log.drain(0..n - 50);
if n > 80 {
s.log.drain(0..n - 80);
}
}
/// A global (install/update) failure — not tied to a single server.
fn fail(&self, msg: impl Into<String>) {
let msg = msg.into();
tracing::warn!(target: "cmf", "{msg}");
self.status.lock().unwrap().log.push(format!("✗ {msg}"));
}
/// Record a fatal error against a specific server (and the log).
fn fail_server(&self, id: &str, msg: impl Into<String>) {
let msg = msg.into();
tracing::warn!(target: "cmf", "[{id}] {msg}");
let mut s = self.status.lock().unwrap();
s.last_error = Some(msg.clone());
s.log.push(format!("✗ {msg}"));
if let Some(sv) = s.servers.iter_mut().find(|x| x.id == id) {
sv.last_error = Some(msg.clone());
}
s.log.push(format!("✗ [{id}] {msg}"));
}
fn set_child(&self, child: Child) {
*self.child.lock().unwrap() = Some(child);
self.status.lock().unwrap().running = true;
/// Seed the per-server status slots from the effective server list.
fn init_servers(&self, servers: &[CmfServer], host: &str) {
let mut s = self.status.lock().unwrap();
s.enabled = true;
s.servers = servers
.iter()
.map(|sv| CmfServerStatus {
id: sv.id.clone(),
model: sv.model.clone(),
port: sv.port,
base_url: format!("http://{host}:{}/v1", sv.port),
running: false,
healthy: false,
last_error: None,
})
.collect();
}
/// Stop the managed server (used on shutdown / reconfigure).
fn set_running(&self, id: &str, running: bool) {
if let Some(sv) = self
.status
.lock()
.unwrap()
.servers
.iter_mut()
.find(|x| x.id == id)
{
sv.running = running;
}
}
fn set_healthy(&self, id: &str, healthy: bool) {
if let Some(sv) = self
.status
.lock()
.unwrap()
.servers
.iter_mut()
.find(|x| x.id == id)
{
sv.healthy = healthy;
}
}
fn add_child(&self, id: String, child: Child) {
self.children.lock().unwrap().push(Managed {
id: id.clone(),
child,
});
self.set_running(&id, true);
}
fn is_running(&self, id: &str) -> bool {
self.status
.lock()
.unwrap()
.servers
.iter()
.any(|s| s.id == id && s.running)
}
/// Stop all managed servers (used on shutdown / reconfigure).
pub async fn stop(&self) {
if let Some(mut child) = self.child.lock().unwrap().take() {
let _ = child.start_kill();
let mut kids = std::mem::take(&mut *self.children.lock().unwrap());
for m in kids.iter_mut() {
let _ = m.child.start_kill();
}
let mut s = self.status.lock().unwrap();
s.running = false;
s.healthy = false;
for sv in s.servers.iter_mut() {
sv.running = false;
sv.healthy = false;
}
}
/// Stop and forget a single managed server (used when a model is deleted).
pub async fn stop_one(&self, id: &str) {
let removed = {
let mut kids = self.children.lock().unwrap();
kids.iter()
.position(|m| m.id == id)
.map(|pos| kids.remove(pos))
};
if let Some(mut m) = removed {
let _ = m.child.start_kill();
}
self.status.lock().unwrap().servers.retain(|sv| sv.id != id);
}
}
@@ -92,7 +192,7 @@ impl Default for CmfRuntime {
fn default() -> Self {
Self {
status: Mutex::new(CmfStatus::default()),
child: Mutex::new(None),
children: Mutex::new(Vec::new()),
}
}
}
@@ -113,14 +213,15 @@ async fn latest_crates_version() -> Option<String> {
.await
.ok()?;
let v: serde_json::Value = serde_json::from_str(&body).ok()?;
v["crate"]["newest_version"]
.as_str()
.map(|s| s.to_string())
v["crate"]["newest_version"].as_str().map(|s| s.to_string())
}
/// `<bin> --version` → the trailing version token (e.g. "cortiq 0.1.2" → "0.1.2").
fn installed_version(bin: &str) -> Option<String> {
let out = std::process::Command::new(bin).arg("--version").output().ok()?;
pub(crate) fn installed_version(bin: &str) -> Option<String> {
let out = std::process::Command::new(bin)
.arg("--version")
.output()
.ok()?;
if !out.status.success() {
return None;
}
@@ -130,7 +231,11 @@ fn installed_version(bin: &str) -> Option<String> {
fn parse_ver(s: &str) -> (u32, u32, u32) {
let mut it = s.split('.').map(|x| x.trim().parse::<u32>().unwrap_or(0));
(it.next().unwrap_or(0), it.next().unwrap_or(0), it.next().unwrap_or(0))
(
it.next().unwrap_or(0),
it.next().unwrap_or(0),
it.next().unwrap_or(0),
)
}
fn version_lt(a: &str, b: &str) -> bool {
@@ -152,24 +257,42 @@ async fn cargo_install(force: bool) -> Result<(), String> {
}
}
/// Spawn `cortiq serve <model> --host <host> --port <port>`; the child is killed
/// when its handle is dropped (i.e. when the gateway shuts down).
fn spawn_serve(cfg: &CmfCfg) -> Result<Child, String> {
/// Install/upgrade `cortiq-cli` from crates.io on demand (the admin "Install
/// cortiq" button). Runs in the background; progress + result land in the CMF
/// status log, and `installed_version` is refreshed on success.
pub async fn install_now(rt: Arc<CmfRuntime>, bin: String) {
rt.log("installing cortiq-cli from crates.io (this compiles — a few minutes)…");
match cargo_install(true).await {
Ok(()) => {
let v = installed_version(&bin);
rt.status.lock().unwrap().installed_version = v.clone();
rt.log(format!(
"installed cortiq-cli {}",
v.as_deref().unwrap_or("?")
));
}
Err(e) => rt.fail(format!("cargo install cortiq-cli failed: {e}")),
}
}
/// Spawn `cortiq serve <model> --host <host> --port <port>` for one server; the
/// child is killed when its handle is dropped (i.e. when the gateway shuts down).
fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
let mut cmd = Command::new(&cfg.cortiq_bin);
cmd.arg("serve")
.arg(&cfg.local_model)
.arg(&server.model)
.arg("--host")
.arg(&cfg.local_host)
.arg("--port")
.arg(cfg.local_port.to_string());
// Parallelise decode matvecs across cores (CMF_THREADS). 0 = leave the
.arg(server.port.to_string());
// Parallelise decode matvecs across cores (CMF_THREADS). 0/1 = leave the
// runtime's own default; N>1 gave ~2.8x on a 15B CPU model.
if cfg.threads > 1 {
cmd.env("CMF_THREADS", cfg.threads.to_string());
if server.threads > 1 {
cmd.env("CMF_THREADS", server.threads.to_string());
}
// Offload large matvecs to the GPU (Metal on macOS). Memory-bandwidth-bound
// on unified memory, so mostly a prefill win; opt-in.
if cfg.gpu {
if server.gpu {
cmd.env("CMF_GPU", "1");
}
cmd.stdout(Stdio::null())
@@ -199,24 +322,17 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
false
}
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn →
/// wait healthy. Safe to call in a background task; it never panics.
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
/// configured local model → wait healthy. Safe to call in a background task; it
/// never panics.
pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
if !cfg.manage_server || cfg.local_model.trim().is_empty() {
return;
}
{
let mut s = rt.status.lock().unwrap();
s.enabled = true;
s.base_url = format!("http://{}:{}/v1", cfg.local_host, cfg.local_port);
s.model = cfg.local_model.clone();
}
if !std::path::Path::new(&cfg.local_model).exists() {
rt.fail(format!("local_model not found: {}", cfg.local_model));
let servers = cfg.effective_servers();
if servers.is_empty() {
return;
}
rt.init_servers(&servers, &cfg.local_host);
// 1. Ensure the `cortiq` binary is available.
// 1. Ensure the `cortiq` binary is available (once for all servers).
let mut ver = installed_version(&cfg.cortiq_bin);
if ver.is_none() {
if cfg.auto_install {
@@ -230,7 +346,9 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
rt.log("installed cortiq-cli");
}
} else {
rt.fail("cortiq binary not found (set [cmf].auto_install = true, or install cortiq-cli)");
rt.fail(
"cortiq binary not found (set [cmf].auto_install = true, or install cortiq-cli)",
);
return;
}
}
@@ -240,8 +358,15 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
if cfg.auto_update {
if let Some(latest) = latest_crates_version().await {
rt.status.lock().unwrap().latest_version = Some(latest.clone());
if ver.as_deref().map(|c| version_lt(c, &latest)).unwrap_or(false) {
rt.log(format!("updating cortiq-cli {} → {latest}…", ver.as_deref().unwrap_or("?")));
if ver
.as_deref()
.map(|c| version_lt(c, &latest))
.unwrap_or(false)
{
rt.log(format!(
"updating cortiq-cli {} → {latest}…",
ver.as_deref().unwrap_or("?")
));
if let Err(e) = cargo_install(true).await {
rt.log(format!("update failed (keeping current): {e}"));
} else {
@@ -253,27 +378,38 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
}
}
// 3. Spawn the local server.
rt.log(format!(
"starting: cortiq serve {} --host {} --port {}",
cfg.local_model, cfg.local_host, cfg.local_port
));
match spawn_serve(&cfg) {
Ok(child) => rt.set_child(child),
Err(e) => {
rt.fail(format!("failed to spawn cortiq serve: {e}"));
return;
// 3. Spawn each server (fast).
for server in &servers {
if !std::path::Path::new(&server.model).exists() {
rt.fail_server(
&server.id,
format!("model file not found: {}", server.model),
);
continue;
}
rt.log(format!(
"starting: cortiq serve {} --host {} --port {}",
server.model, cfg.local_host, server.port
));
match spawn_serve(server, &cfg) {
Ok(child) => rt.add_child(server.id.clone(), child),
Err(e) => rt.fail_server(&server.id, format!("failed to spawn: {e}")),
}
}
// 4. Wait until it is serving (model load can take a while for big models).
if wait_healthy(&cfg.local_host, cfg.local_port, 240).await {
rt.status.lock().unwrap().healthy = true;
rt.log(format!(
"local CMF server ready — registered as model '{}' at http://{}:{}/v1",
cfg.model_id, cfg.local_host, cfg.local_port
));
} else {
rt.fail("local CMF server did not become healthy in time");
// 4. Wait until each spawned server is serving (model load can be slow).
for server in &servers {
if !rt.is_running(&server.id) {
continue; // failed to spawn / file missing
}
if wait_healthy(&cfg.local_host, server.port, 240).await {
rt.set_healthy(&server.id, true);
rt.log(format!(
"local CMF server ready — '{}' at http://{}:{}/v1",
server.id, cfg.local_host, server.port
));
} else {
rt.fail_server(&server.id, "did not become healthy in time");
}
}
}
+75 -6
View File
@@ -10,10 +10,18 @@ use std::path::PathBuf;
/// `~/.cortiq-gateway` — so defaults work regardless of the process CWD
/// (important once the gateway is installed rather than run from its repo).
pub fn data_dir() -> PathBuf {
for (var, sub) in [("CORTIQ_GATEWAY_HOME", ""), ("XDG_DATA_HOME", "cortiq-gateway"), ("HOME", ".cortiq-gateway")] {
for (var, sub) in [
("CORTIQ_GATEWAY_HOME", ""),
("XDG_DATA_HOME", "cortiq-gateway"),
("HOME", ".cortiq-gateway"),
] {
if let Ok(v) = std::env::var(var) {
if !v.is_empty() {
return if sub.is_empty() { PathBuf::from(v) } else { PathBuf::from(v).join(sub) };
return if sub.is_empty() {
PathBuf::from(v)
} else {
PathBuf::from(v).join(sub)
};
}
}
}
@@ -59,6 +67,24 @@ pub struct Config {
pub cmf: CmfCfg,
}
/// One managed local model: the gateway runs a dedicated `cortiq serve` for it
/// on its own port and registers it in the pool under `id`.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct CmfServer {
/// Pool id it is registered under (e.g. "cmf-qwen3-5-2b").
pub id: String,
/// Path to the `.cmf` to serve.
pub model: String,
/// TCP port the dedicated `cortiq serve` binds to (must be unique).
pub port: u16,
/// Worker threads (CMF_THREADS). 0 = runtime default.
#[serde(default = "default_cmf_threads")]
pub threads: usize,
/// Offload matvecs to the GPU (CMF_GPU=1).
#[serde(default)]
pub gpu: bool,
}
/// CMF-format model factory: import a HuggingFace model, convert it to a
/// local `.cmf` (quantized), and serve it via cortiq-server. Points the
/// gateway at the Python converter + runtime on the host.
@@ -121,6 +147,35 @@ pub struct CmfCfg {
/// (helps prefill of long prompts). Off by default.
#[serde(default)]
pub gpu: bool,
/// Managed local models. When non-empty (and `manage_server` is on), the
/// gateway spawns and supervises one `cortiq serve` per entry — each on its
/// own port — and registers each in the pool. Empty = fall back to the single
/// legacy `local_model` / `local_port` / `model_id` below (back-compat).
#[serde(default)]
pub servers: Vec<CmfServer>,
}
impl CmfCfg {
/// The managed local models actually in effect: the `servers` list when set,
/// otherwise the single legacy `local_model` (as a one-element list). Empty
/// when `manage_server` is off or nothing is configured.
pub fn effective_servers(&self) -> Vec<CmfServer> {
if !self.manage_server {
return Vec::new();
}
if !self.servers.is_empty() {
return self.servers.clone();
}
if !self.local_model.trim().is_empty() {
return vec![CmfServer {
id: self.model_id.clone(),
model: self.local_model.clone(),
port: self.local_port,
threads: self.threads,
gpu: self.gpu,
}];
}
Vec::new()
}
}
impl Default for CmfCfg {
fn default() -> Self {
@@ -141,6 +196,7 @@ impl Default for CmfCfg {
local_only: false,
threads: default_cmf_threads(),
gpu: false,
servers: Vec::new(),
}
}
}
@@ -597,9 +653,15 @@ impl Config {
// A managed local CMF model counts as an available model even though it
// is registered dynamically rather than via a [[models]] entry — this
// is what lets the gateway run on local models alone (no cloud, no router).
let has_local = self.cmf.manage_server && !self.cmf.local_model.trim().is_empty();
let servers = self.cmf.effective_servers();
let has_local = !servers.is_empty();
if self.models.is_empty() && !has_local {
anyhow::bail!("config: at least one [[models]] entry (or a managed local CMF model) is required");
// Fresh install: start with an EMPTY model pool so the admin console
// comes up — the user adds models in /admin → Models (or points [cmf]
// at a local .cmf). Routing simply has nothing to select until then.
tracing::warn!(
"no models configured — starting with an empty pool; add one in /admin → Models"
);
}
// uniqueness of model ids
let mut seen = std::collections::HashSet::new();
@@ -611,8 +673,15 @@ impl Config {
// every target in [routing] must exist in the model pool
let mut ids: std::collections::HashSet<&str> =
self.models.iter().map(|m| m.id.as_str()).collect();
if has_local {
ids.insert(self.cmf.model_id.as_str());
// managed local servers are known ids too; also enforce unique ports/ids
let mut ports = std::collections::HashSet::new();
for s in &servers {
if !ids.insert(s.id.as_str()) {
anyhow::bail!("config: duplicate managed model id '{}'", s.id);
}
if !ports.insert(s.port) {
anyhow::bail!("config: two managed models share port {}", s.port);
}
}
if !self.routing.default.is_empty() && !ids.contains(self.routing.default.as_str()) {
anyhow::bail!(
+115 -32
View File
@@ -31,13 +31,19 @@ fn cleanup_output(path: &str) {
}
fn now() -> u64 {
SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_secs()
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// Job id from a hash of (repo, nanos) — unique without a uuid dependency.
fn gen_id(repo: &str) -> String {
use std::hash::{Hash, Hasher};
let nanos = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_nanos();
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_nanos();
let mut h = std::collections::hash_map::DefaultHasher::new();
repo.hash(&mut h);
nanos.hash(&mut h);
@@ -82,8 +88,8 @@ pub struct Job {
pub started: u64,
pub finished: Option<u64>,
pub size_bytes: Option<u64>,
pub progress: Option<f32>, // 0..1, parsed from converter @PROGRESS
pub phase: Option<String>, // current phase label
pub progress: Option<f32>, // 0..1, parsed from converter @PROGRESS
pub phase: Option<String>, // current phase label
}
#[derive(Default)]
@@ -178,16 +184,24 @@ impl JobStore {
}
pub fn list(&self) -> Vec<Job> {
let mut v: Vec<Job> = self.jobs.lock().unwrap().values().cloned().collect();
v.sort_by(|a, b| b.started.cmp(&a.started));
v.sort_by_key(|x| std::cmp::Reverse(x.started));
v.truncate(20);
v
}
}
/// Proxy HuggingFace model search (avoids browser CORS + adds our token).
pub async fn hf_search(query: &str, limit: usize, token: Option<&str>) -> Result<serde_json::Value, String> {
pub async fn hf_search(
query: &str,
limit: usize,
token: Option<&str>,
) -> Result<serde_json::Value, String> {
let q: String = url_escape(query);
let sort = if query.trim().is_empty() { "trendingScore" } else { "downloads" };
let sort = if query.trim().is_empty() {
"trendingScore"
} else {
"downloads"
};
let url = format!(
"https://huggingface.co/api/models?search={q}&sort={sort}&direction=-1&limit={limit}&full=false"
);
@@ -218,11 +232,28 @@ fn url_escape(s: &str) -> String {
fn sanitize(s: &str) -> String {
s.chars()
.map(|c| if c.is_ascii_alphanumeric() || c == '-' || c == '_' { c } else { '-' })
.map(|c| {
if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
c
} else {
'-'
}
})
.collect()
}
/// Kick off a conversion. Returns the job id immediately; progress streams
/// Map the UI quant label to the native `cortiq` quant flag.
fn map_quant(q: &str) -> &'static str {
match q.to_ascii_uppercase().as_str() {
s if s.starts_with("Q4") => "q4",
"Q8_2F" => "q8_2f",
"F16" | "FP16" => "f16",
"VBIT" => "vbit",
_ => "q8",
}
}
/// into the job log. Fails fast if the converter script is missing.
pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Result<String, String> {
if p.repo.trim().is_empty() {
@@ -233,7 +264,10 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
// doesn't support yet (linear-attention folding, v-bit shaping).
let use_python = p.linear_core.is_some() || p.vbit_shape.is_some() || p.mean_bits.is_some();
if use_python && !std::path::Path::new(&cfg.converter).exists() {
return Err(format!("advanced options need the Python converter, not found: {}", cfg.converter));
return Err(format!(
"advanced options need the Python converter, not found: {}",
cfg.converter
));
}
let _ = std::fs::create_dir_all(&cfg.models_dir);
let base = if p.name.trim().is_empty() {
@@ -277,35 +311,84 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
let (program, args, workdir): (String, Vec<String>, std::path::PathBuf) = if use_python {
let conv = std::path::Path::new(&cfg.converter);
let mut a = vec![
std::fs::canonicalize(conv).map(|p| p.to_string_lossy().to_string()).unwrap_or_else(|_| cfg.converter.clone()),
"--model".into(), p.repo.clone(),
"--quant".into(), p.quant.clone(),
"--output".into(), output_abs.clone(),
std::fs::canonicalize(conv)
.map(|p| p.to_string_lossy().to_string())
.unwrap_or_else(|_| cfg.converter.clone()),
"--model".into(),
p.repo.clone(),
"--quant".into(),
p.quant.clone(),
"--output".into(),
output_abs.clone(),
];
if let Some(lc) = &p.linear_core { a.push("--linear-core".into()); a.push(lc.clone()); }
if let Some(n) = p.nphase { a.push("--nphase".into()); a.push(n.to_string()); }
if let Some(vs) = &p.vbit_shape { a.push("--vbit-shape".into()); a.push(vs.clone()); }
if let Some(mb) = p.mean_bits { a.push("--mean-bits".into()); a.push(mb.to_string()); }
if let Some(g) = p.shard_max_gb { a.push("--shard-max-gb".into()); a.push(g.to_string()); }
if p.skip_mtp { a.push("--skip-mtp".into()); }
let wd = conv.parent().map(|d| d.to_path_buf()).unwrap_or_else(|| std::path::PathBuf::from("."));
if let Some(lc) = &p.linear_core {
a.push("--linear-core".into());
a.push(lc.clone());
}
if let Some(n) = p.nphase {
a.push("--nphase".into());
a.push(n.to_string());
}
if let Some(vs) = &p.vbit_shape {
a.push("--vbit-shape".into());
a.push(vs.clone());
}
if let Some(mb) = p.mean_bits {
a.push("--mean-bits".into());
a.push(mb.to_string());
}
if let Some(g) = p.shard_max_gb {
a.push("--shard-max-gb".into());
a.push(g.to_string());
}
if p.skip_mtp {
a.push("--skip-mtp".into());
}
let wd = conv
.parent()
.map(|d| d.to_path_buf())
.unwrap_or_else(|| std::path::PathBuf::from("."));
(cfg.python_bin.clone(), a, wd)
} else if p.repo.to_ascii_lowercase().contains("gguf") {
// GGUF repo → native `cortiq import-gguf` (downloads + dequantizes any
// common ggml quant type: Q4_0/1, Q5_0/1, Q8_0, Q2_K..Q6_K, IQ4_NL/XS).
let mut a = vec![
"import-gguf".into(),
p.repo.clone(),
"--quant".into(),
map_quant(&p.quant).into(),
"--output".into(),
output_abs.clone(),
];
if let Some(t) = &hf_token {
a.push("--hf-token".into());
a.push(t.clone());
}
(
cfg.cortiq_bin.clone(),
a,
std::env::current_dir().unwrap_or_else(|_| std::path::PathBuf::from(".")),
)
} else {
// native Rust converter — quant name maps to cortiq's scheme.
let quant = match p.quant.to_ascii_uppercase().as_str() {
s if s.starts_with("Q4") => "q4",
"Q8_2F" => "q8_2f",
"F16" | "FP16" => "f16",
_ => "q8",
};
// safetensors → native `cortiq convert` (dense / MoE / GatedDeltaNet).
let mut a = vec![
"convert".into(),
"--model".into(), p.repo.clone(),
"--quant".into(), quant.into(),
"--output".into(), output_abs.clone(),
"--model".into(),
p.repo.clone(),
"--quant".into(),
map_quant(&p.quant).into(),
"--output".into(),
output_abs.clone(),
];
if let Some(t) = &hf_token { a.push("--hf-token".into()); a.push(t.clone()); }
(cfg.cortiq_bin.clone(), a, std::env::current_dir().unwrap_or_else(|_| std::path::PathBuf::from(".")))
if let Some(t) = &hf_token {
a.push("--hf-token".into());
a.push(t.clone());
}
(
cfg.cortiq_bin.clone(),
a,
std::env::current_dir().unwrap_or_else(|_| std::path::PathBuf::from(".")),
)
};
let ret_id = id.clone();
+44 -12
View File
@@ -208,15 +208,25 @@ impl Pipeline {
};
let (decision, route_source, candidates) = if live.cfg.cmf.local_only {
(mk("local-only", "local"), "local".to_string(), live.local_candidates())
(
mk("local-only", "local"),
"local".to_string(),
live.local_candidates(),
)
} else {
match &req.routing {
RoutingDirective::Pinned { model_id } => {
(mk("pinned", "pinned"), "pinned".to_string(), vec![model_id.clone()])
}
RoutingDirective::Pinned { model_id } => (
mk("pinned", "pinned"),
"pinned".to_string(),
vec![model_id.clone()],
),
RoutingDirective::Auto { profile } => {
if !live.cfg.router.enabled {
(mk("router-disabled", "local"), "local".to_string(), live.local_candidates())
(
mk("router-disabled", "local"),
"local".to_string(),
live.local_candidates(),
)
} else {
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
match live.router.route(&text, prof).await {
@@ -229,7 +239,11 @@ impl Pipeline {
tracing::warn!(
"Router is unavailable. Gracefully degrading to the local/default model."
);
(mk("degraded", "fallback"), "fallback".to_string(), live.local_candidates())
(
mk("degraded", "fallback"),
"fallback".to_string(),
live.local_candidates(),
)
}
}
}
@@ -253,9 +267,21 @@ impl Pipeline {
}
}
// Local-first safety net: if routing produced no usable candidate (e.g.
// an unknown pinned model while the router is off), fall back to the
// managed local model so a configured local backend still answers.
// An explicit pin must never silently answer from a different model: if
// the pinned id is unknown or was filtered out, fail loudly so the caller
// knows their choice was not honoured (instead of getting a stray reply
// from the managed local model via the safety net below).
if final_candidates.is_empty() {
if let RoutingDirective::Pinned { model_id } = &req.routing {
return Err(GatewayError::UpstreamUnavailable(format!(
"pinned model '{model_id}' is not available — it is not in the pool (deleted?) or lacks a required capability such as tools"
)));
}
}
// Local-first safety net: for AUTO/degraded routes only, if routing
// produced no usable candidate, fall back to the managed local model so a
// configured local backend still answers.
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
for id in live.local_candidates() {
if let Some(prov) = live.registry.get(&id) {
@@ -289,7 +315,10 @@ impl Pipeline {
&& !sh.local_model_id.is_empty()
&& matches!(req.routing, RoutingDirective::Auto { .. })
&& state.promotion.serves_local(&decision.task_label)
&& !matches!(decision.complexity_tier.as_str(), "high" | "hard" | "complex")
&& !matches!(
decision.complexity_tier.as_str(),
"high" | "hard" | "complex"
)
&& !final_candidates.iter().any(|m| m == &sh.local_model_id)
{
final_candidates.insert(0, sh.local_model_id.clone());
@@ -713,7 +742,7 @@ fn parse_judge(text: &str) -> (u8, bool) {
let score = text
.split("score")
.nth(1)
.and_then(|t| t.chars().skip_while(|c| !c.is_ascii_digit()).next())
.and_then(|t| t.chars().find(|c| c.is_ascii_digit()))
.and_then(|c| c.to_digit(10))
.map(|d| d.min(4) as u8);
let hard_fail = text
@@ -747,7 +776,10 @@ async fn shadow_capture(
};
// 1. Local answer (free, CPU).
let a_local = match local.chat(shadow_req(&cfg.local_model_id, clip(&prompt, 4000), 512)).await {
let a_local = match local
.chat(shadow_req(&cfg.local_model_id, clip(&prompt, 4000), 512))
.await
{
Ok(r) => r
.choices
.into_iter()
+96 -30
View File
@@ -18,7 +18,10 @@ use std::sync::Mutex;
use std::time::{SystemTime, UNIX_EPOCH};
fn now_secs() -> u64 {
SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_secs()
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// One judgment of a local answer (JSONL line, replayed on startup).
@@ -62,14 +65,14 @@ pub enum PromotionState {
#[derive(Clone, Debug)]
pub struct PromotionCfg {
pub enabled: bool,
pub window: usize, // rolling judgments per label (W)
pub n_min: usize, // min judgments before promotion is even considered
pub promote_lb: f64, // Wilson-95 LB of pass must clear this
pub demote_pass: f64, // point pass-rate below this → demote a served label
pub hardfail_ub_max: f64,// Wilson-95 UPPER bound of hard_fail must stay under this
pub soak: usize, // extra Canary judgments before LocalServed
pub z: f64, // 1.96 = 95%
pub file: Option<String>,// judge.jsonl path (append + replay)
pub window: usize, // rolling judgments per label (W)
pub n_min: usize, // min judgments before promotion is even considered
pub promote_lb: f64, // Wilson-95 LB of pass must clear this
pub demote_pass: f64, // point pass-rate below this → demote a served label
pub hardfail_ub_max: f64, // Wilson-95 UPPER bound of hard_fail must stay under this
pub soak: usize, // extra Canary judgments before LocalServed
pub z: f64, // 1.96 = 95%
pub file: Option<String>, // judge.jsonl path (append + replay)
}
impl Default for PromotionCfg {
@@ -92,7 +95,7 @@ struct LabelState {
passes: VecDeque<bool>, // rolling window of pass/fail
hard_fails: VecDeque<bool>, // rolling window of hard_fail flags
state: PromotionState,
soaked: usize, // judgments accumulated while in Canary
soaked: usize, // judgments accumulated while in Canary
}
impl LabelState {
@@ -124,20 +127,29 @@ pub fn wilson_bound(s: usize, n: usize, z: f64, upper: bool) -> f64 {
let denom = 1.0 + z2 / n;
let centre = p + z2 / (2.0 * n);
let margin = z * ((p * (1.0 - p) / n) + z2 / (4.0 * n * n)).sqrt();
let v = if upper { centre + margin } else { centre - margin };
let v = if upper {
centre + margin
} else {
centre - margin
};
(v / denom).clamp(0.0, 1.0)
}
impl Promotion {
pub fn new(cfg: PromotionCfg) -> std::sync::Arc<Self> {
let p = std::sync::Arc::new(Self { cfg, labels: Mutex::new(HashMap::new()) });
let p = std::sync::Arc::new(Self {
cfg,
labels: Mutex::new(HashMap::new()),
});
p.replay();
p
}
fn replay(&self) {
let Some(path) = &self.cfg.file else { return };
let Ok(content) = std::fs::read_to_string(path) else { return };
let Ok(content) = std::fs::read_to_string(path) else {
return;
};
for line in content.lines() {
if let Ok(rec) = serde_json::from_str::<JudgeRecord>(line) {
self.apply(&rec);
@@ -151,8 +163,10 @@ impl Promotion {
let st = self.apply(&rec);
if let Some(path) = &self.cfg.file {
if let Ok(line) = serde_json::to_string(&rec) {
if let Ok(mut f) =
std::fs::OpenOptions::new().create(true).append(true).open(path)
if let Ok(mut f) = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(path)
{
let _ = writeln!(f, "{line}");
}
@@ -163,7 +177,9 @@ impl Promotion {
fn apply(&self, rec: &JudgeRecord) -> PromotionState {
let mut map = self.labels.lock().unwrap();
let ls = map.entry(rec.task_label.clone()).or_insert_with(LabelState::new);
let ls = map
.entry(rec.task_label.clone())
.or_insert_with(LabelState::new);
push_cap(&mut ls.passes, rec.pass, self.cfg.window);
push_cap(&mut ls.hard_fails, rec.hard_fail, self.cfg.window);
self.advance(ls);
@@ -188,7 +204,12 @@ impl Promotion {
ls.state = match ls.state {
PromotionState::CloudOnly | PromotionState::ShadowJudged => {
if bar { ls.soaked = 0; PromotionState::Canary } else { PromotionState::ShadowJudged }
if bar {
ls.soaked = 0;
PromotionState::Canary
} else {
PromotionState::ShadowJudged
}
}
PromotionState::Canary => {
if regressed {
@@ -203,7 +224,11 @@ impl Promotion {
}
}
PromotionState::LocalServed => {
if regressed { PromotionState::Demoted } else { PromotionState::LocalServed }
if regressed {
PromotionState::Demoted
} else {
PromotionState::LocalServed
}
}
// Demoted goes back to SHADOW to re-earn promotion (skill stays in .cmf).
PromotionState::Demoted => PromotionState::ShadowJudged,
@@ -223,7 +248,10 @@ impl Promotion {
/// Should a request of this label be SERVED by the local model?
/// (Canary/LocalServed; caller still applies the per-request veto.)
pub fn serves_local(&self, label: &str) -> bool {
matches!(self.state(label), PromotionState::Canary | PromotionState::LocalServed)
matches!(
self.state(label),
PromotionState::Canary | PromotionState::LocalServed
)
}
/// Snapshot for /admin: (state, n, pass-rate, Wilson-LB) per label.
@@ -235,10 +263,16 @@ impl Promotion {
let n = ls.passes.len();
let s = ls.passes.iter().filter(|&&p| p).count();
let p = if n > 0 { s as f64 / n as f64 } else { 0.0 };
(k.clone(), ls.state, n, p, wilson_bound(s, n, self.cfg.z, false))
(
k.clone(),
ls.state,
n,
p,
wilson_bound(s, n, self.cfg.z, false),
)
})
.collect();
out.sort_by(|a, b| b.2.cmp(&a.2));
out.sort_by_key(|x| std::cmp::Reverse(x.2));
out
}
@@ -259,12 +293,24 @@ mod tests {
use super::*;
fn cfg() -> PromotionCfg {
PromotionCfg { enabled: true, n_min: 200, soak: 0, ..Default::default() }
PromotionCfg {
enabled: true,
n_min: 200,
soak: 0,
..Default::default()
}
}
fn rec(label: &str, pass: bool, hard_fail: bool) -> JudgeRecord {
JudgeRecord {
ts: 0, task_label: label.into(), pass, hard_fail, score: if pass { 4 } else { 0 },
source: "checker".into(), recon_e: None, born_conf: None, judge_tokens: 0,
ts: 0,
task_label: label.into(),
pass,
hard_fail,
score: if pass { 4 } else { 0 },
source: "checker".into(),
recon_e: None,
born_conf: None,
judge_tokens: 0,
}
}
@@ -316,7 +362,10 @@ mod tests {
match row {
Some((_, st, n, pr, lb)) => println!(
" {label:<12} {:<13} n={n:<4} pass={:.0}% LB={:.3} serves_local={}",
format!("{st:?}"), pr * 100.0, lb, p.serves_local(label)
format!("{st:?}"),
pr * 100.0,
lb,
p.serves_local(label)
),
None => println!(" {label:<12} (cloud-only, no judgments)"),
}
@@ -324,19 +373,29 @@ mod tests {
println!("\n── self-warming loop, live ──");
// A mature skill (e.g. ru-tech): local answers pass the judge ~99%.
println!(" 1) SHADOW→CANARY: судим локаль 'ru_tech' (сильный скилл ~99% pass), 210 судейств:");
println!(
" 1) SHADOW→CANARY: судим локаль 'ru_tech' (сильный скилл ~99% pass), 210 судейств:"
);
for i in 0..210 {
p.record(rec("ru_tech", i % 100 != 0, false));
}
show("ru_tech");
assert_eq!(p.state("ru_tech"), PromotionState::Canary, "n≥200 & LB≥0.95 → CANARY");
assert_eq!(
p.state("ru_tech"),
PromotionState::Canary,
"n≥200 & LB≥0.95 → CANARY"
);
println!(" 2) CANARY soak (ещё 60 судейств держат планку) → LOCAL_SERVED:");
for i in 0..60 {
p.record(rec("ru_tech", i % 100 != 0, false));
}
show("ru_tech");
assert_eq!(p.state("ru_tech"), PromotionState::LocalServed, "soak → LOCAL_SERVED");
assert_eq!(
p.state("ru_tech"),
PromotionState::LocalServed,
"soak → LOCAL_SERVED"
);
assert!(p.serves_local("ru_tech"), "теперь обслуживается локально");
// The code dataset: local fails the checker → never promotes.
@@ -345,7 +404,11 @@ mod tests {
p.record(rec("code", false, false));
}
show("code");
assert_eq!(p.state("code"), PromotionState::ShadowJudged, "остаётся на облаке — безопасно");
assert_eq!(
p.state("code"),
PromotionState::ShadowJudged,
"остаётся на облаке — безопасно"
);
assert!(!p.serves_local("code"));
// Drift on the promoted label → auto-demote back to cloud.
@@ -376,6 +439,9 @@ mod tests {
for _ in 0..300 {
p2.record(rec("t", false, false)); // quality collapses
}
assert!(matches!(p2.state("t"), PromotionState::Demoted | PromotionState::ShadowJudged));
assert!(matches!(
p2.state("t"),
PromotionState::Demoted | PromotionState::ShadowJudged
));
}
}
+9 -9
View File
@@ -28,17 +28,17 @@ impl Registry {
by_id.insert(m.id.clone(), provider);
}
// Managed local CMF server: register it as an OpenAI-compatible provider
// so it can be routed to like any other backend — including when the
// router is unavailable or the gateway is in local-only mode.
if cfg.cmf.manage_server
&& !cfg.cmf.local_model.trim().is_empty()
&& !by_id.contains_key(&cfg.cmf.model_id)
{
// Managed local CMF servers: register each as an OpenAI-compatible
// provider so it can be routed to like any other backend — including when
// the router is unavailable or the gateway is in local-only mode.
for s in cfg.cmf.effective_servers() {
if by_id.contains_key(&s.id) {
continue; // a static [[models]] entry with the same id wins
}
let m = ModelCfg {
id: cfg.cmf.model_id.clone(),
id: s.id.clone(),
provider: "openai".into(),
base_url: format!("http://{}:{}/v1", cfg.cmf.local_host, cfg.cmf.local_port),
base_url: format!("http://{}:{}/v1", cfg.cmf.local_host, s.port),
model: "cortiq".into(),
cost_tier: "local".into(),
price_in: 0.0,
+7 -3
View File
@@ -49,8 +49,8 @@ impl Live {
/// (local-only / disabled) or unavailable, so CMF models serve with no router.
pub fn local_candidates(&self) -> Vec<String> {
let mut v = Vec::new();
if self.cfg.cmf.manage_server && !self.cfg.cmf.local_model.trim().is_empty() {
v.push(self.cfg.cmf.model_id.clone());
for s in self.cfg.cmf.effective_servers() {
v.push(s.id);
}
let def = self.routing.default_model().to_string();
if !def.is_empty() && !v.contains(&def) {
@@ -90,7 +90,11 @@ fn promotion_cfg(s: &crate::config::ShadowCfg) -> crate::promotion::PromotionCfg
n_min: s.n_min,
promote_lb: s.promote_lb,
soak: s.soak,
file: if s.file.is_empty() { None } else { Some(s.file.clone()) },
file: if s.file.is_empty() {
None
} else {
Some(s.file.clone())
},
..Default::default()
}
}
+90
View File
@@ -177,6 +177,18 @@ textarea { resize: vertical; min-height: 64px; }
.check input { width: auto; }
.hint { color: var(--faint); font-size: 12px; margin-top: 4px; }
/* "?" help badge — hover/focus shows the native title tooltip explaining a setting */
.help {
display: inline-flex; align-items: center; justify-content: center;
width: 15px; height: 15px; border-radius: 50%;
border: 1px solid var(--border); background: var(--panel);
color: var(--muted); font-size: 10px; font-weight: 700; line-height: 1;
cursor: help; vertical-align: middle; user-select: none;
transition: 0.12s;
}
.help:hover, .help:focus { color: var(--accent); border-color: var(--accent); outline: none; }
label.field > span .help { text-transform: none; }
/* ---- buttons ------------------------------------------------------------- */
.btn {
display: inline-flex; align-items: center; justify-content: center; gap: 8px;
@@ -307,3 +319,81 @@ tbody tr:hover td { background: var(--hover); }
.callout { border: 1px solid var(--border); border-left: 3px solid var(--warn); background: color-mix(in srgb, var(--warn) 6%, transparent); border-radius: 10px; padding: 13px 15px; color: var(--muted); margin-bottom: 16px; }
.callout.info { border-left-color: var(--accent-2); background: color-mix(in srgb, var(--accent-2) 6%, transparent); }
.callout b { color: var(--text); }
/* ── Import model (HuggingFace → CMF) ── */
.view-head { margin-bottom: 14px; }
.view-head h2 { margin: 0 0 2px; }
.card-title { font-weight: 600; margin-bottom: 10px; }
.actions { margin-top: 12px; display: flex; gap: 8px; }
.pad { padding: 18px; }
.err { color: #e5534b; }
.btn.sm { padding: 4px 10px; font-size: 12px; }
.hf-search {
width: 100%; box-sizing: border-box; margin-bottom: 12px;
padding: 10px 12px; font-size: 15px;
border: 1px solid var(--border, #2a2f3a); border-radius: 8px;
background: var(--bg-2, #12151c); color: inherit;
}
.hf-grid {
display: grid; grid-template-columns: repeat(auto-fill, minmax(230px, 1fr));
gap: 10px; max-height: 420px; overflow-y: auto;
}
.hf-card {
border: 1px solid var(--border, #2a2f3a); border-radius: 8px;
padding: 10px 12px; cursor: pointer; transition: border-color .12s, background .12s;
background: var(--bg-2, #12151c);
}
.hf-card:hover { border-color: var(--accent, #5b8def); }
.hf-card.sel { border-color: var(--accent, #5b8def); background: rgba(91,141,239,.10); }
.hf-id { font-weight: 600; font-size: 13px; word-break: break-all; margin-bottom: 6px; }
.hf-meta { display: flex; gap: 8px; align-items: center; flex-wrap: wrap; font-size: 12px; }
.badge.bad { background: rgba(229,83,75,.15); color: #e5534b; }
.import-form { margin-top: 14px; }
.adv { margin-top: 8px; }
.adv-body { margin-top: 10px; padding-top: 10px; border-top: 1px dashed var(--border, #2a2f3a); }
.pbar {
height: 8px; border-radius: 6px; overflow: hidden; margin: 10px 0;
background: var(--bg-2, #12151c); border: 1px solid var(--border, #2a2f3a);
}
.pbar-fill { height: 100%; width: 100%; border-radius: 6px; }
.pbar-fill.run {
width: 40%;
background: linear-gradient(90deg, transparent, var(--accent, #5b8def), transparent);
animation: pbar-slide 1.2s infinite ease-in-out;
}
.pbar-fill.ok { background: #3fb950; }
.pbar-fill.err { background: #e5534b; }
@keyframes pbar-slide { 0% { margin-left: -40%; } 100% { margin-left: 100%; } }
.conv-log {
margin-top: 10px; max-height: 260px; overflow-y: auto;
background: var(--bg-2, #0d1017); border: 1px solid var(--border, #2a2f3a);
border-radius: 8px; padding: 10px 12px;
font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px;
white-space: pre-wrap; word-break: break-word; color: var(--muted, #9aa4b2);
}
/* Router-key field: input grows, inline "Save key" button sits beside it */
.key-row { display: flex; gap: 8px; align-items: stretch; }
.key-row input { flex: 1 1 auto; min-width: 0; }
.key-row .btn { flex: 0 0 auto; white-space: nowrap; }
/* Determinate progress fill (known %); .run remains the indeterminate slider */
.pbar-fill.det {
background: linear-gradient(90deg, var(--accent, #5b8def), #7aa2f7);
transition: width .4s ease;
}
/* Import job cards (multiple conversions, live) */
.job { padding: 12px 0; border-top: 1px solid var(--border-soft, #232833); }
.job:first-of-type { border-top: 0; }
.job-head { display: flex; align-items: center; justify-content: space-between; gap: 12px; }
.job-head b { word-break: break-all; }
.small { font-size: 12px; }
/* Router usage: readable, wrapping key/value rows (no horizontal overflow) */
.usage-box { max-width: 100%; overflow-x: hidden; }
.mono.kv { display: flex; gap: 6px; padding: 1px 0; white-space: normal; overflow-wrap: anywhere; word-break: break-word; }
.mono.kv .kv-k { color: var(--muted, #9aa4b2); flex: 0 0 auto; }
.mono.kv .kv-v { font-weight: 600; overflow-wrap: anywhere; word-break: break-word; min-width: 0; }
+22
View File
@@ -50,6 +50,7 @@ export const api = {
// meta / health
meta: () => j("GET", "/meta"),
health: () => j("GET", "/health"),
probeRouter: () => j("POST", "/router/probe"),
// config
getConfig: () => j("GET", "/config"),
@@ -90,6 +91,27 @@ export const api = {
// playground
test: (b) => j("POST", "/test", b),
// shadow / self-warming loop
shadow: () => j("GET", "/shadow"),
// CMF runtime (the `cortiq` CLI that runs the .cmf format)
cmfStatus: () => j("GET", "/cmf"),
cmfInstall: () => j("POST", "/cmf/install"),
cmfPortCheck: (port) => j("GET", "/cmf/port?port=" + encodeURIComponent(port)),
cmfFiles: () => j("GET", "/cmf/files"),
// CMF model factory (HuggingFace → local .cmf)
hfSearch: (q, limit = 24) =>
j("GET", `/hf/search?q=${encodeURIComponent(q || "")}&limit=${limit}`),
startImport: (b) => j("POST", "/import", b),
listImports: () => j("GET", "/import"),
importStatus: (job) => j("GET", "/import/" + encodeURIComponent(job)),
cancelImport: (job) =>
j("POST", "/import/" + encodeURIComponent(job) + "/cancel"),
deleteImport: (job) => j("DELETE", "/import/" + encodeURIComponent(job)),
registerImport: (job) =>
j("POST", "/import/" + encodeURIComponent(job) + "/register"),
};
// Streaming playground: POST /test/stream, calls onDelta(text) per content chunk,
+24 -2
View File
@@ -5,6 +5,7 @@ import { api, getToken, setToken, setUnauthorizedHandler } from "./api.js";
import { renderLogin } from "./views/login.js";
import { renderDashboard } from "./views/dashboard.js";
import { renderModels } from "./views/models.js";
import { renderImport } from "./views/import.js";
import { renderRouting } from "./views/routing.js";
import { renderProtocols } from "./views/protocols.js";
import { renderKeys } from "./views/keys.js";
@@ -14,6 +15,7 @@ import { renderSettings } from "./views/settings.js";
const ROUTES = [
{ id: "dashboard", icon: "▤", view: renderDashboard },
{ id: "models", icon: "▦", view: renderModels },
{ id: "import", icon: "⬇", view: renderImport },
{ id: "routing", icon: "⇄", view: renderRouting },
{ id: "protocols", icon: "⇆", view: renderProtocols },
{ id: "keys", icon: "🔑", view: renderKeys },
@@ -23,6 +25,9 @@ const ROUTES = [
export const appState = { meta: null, health: null, version: "" };
// allaigate site — where router keys are issued and topped up
export const SITE_URL = "https://api.allaigate.com/";
function currentRoute() {
const id = location.hash.replace(/^#\/?/, "") || "dashboard";
return ROUTES.find((r) => r.id === id) || ROUTES[0];
@@ -53,6 +58,13 @@ function sidebar(activeId) {
)
),
h("div", { class: "nav-spacer" }),
h(
"a",
{ class: "nav-item", href: SITE_URL, target: "_blank", rel: "noopener" },
h("span", { class: "ico" }, "◈"),
t("nav.billing"),
h("span", { style: "margin-left:auto;opacity:.55" }, "↗")
),
h("div", { class: "nav-foot" }, t("nav.foot", { v: appState.version || "—" }))
);
}
@@ -61,6 +73,16 @@ function statusPills() {
const h_ = appState.health;
const gwOk = !!appState.meta;
const routerOk = h_?.router?.reachable;
// reachable but no key resolves → warn: routing silently degrades to default
const keyMissing = h_?.router?.key_source === "missing";
const routerDot = h_ ? (routerOk ? (keyMissing ? "warn" : "ok") : "bad") : "warn";
const routerText = h_
? routerOk
? keyMissing
? t("status.noKey")
: t("status.online")
: t("status.offline")
: t("status.checking");
return h(
"div",
{ class: "flex" },
@@ -73,8 +95,8 @@ function statusPills() {
h(
"span",
{ class: "pill", title: h_?.router?.url || "" },
h("span", { class: "dot " + (h_ ? (routerOk ? "ok" : "bad") : "warn") }),
t("status.router") + ": " + (h_ ? (routerOk ? t("status.online") : t("status.offline")) : t("status.checking"))
h("span", { class: "dot " + routerDot }),
t("status.router") + ": " + routerText
)
);
}
+15
View File
@@ -33,6 +33,7 @@ export default {
"nav.playground": "Playground",
"nav.settings": "Einstellungen",
"nav.foot": "Gateway · v{v}",
"nav.billing": "Router-Key & Abrechnung",
// topbar / status
"status.router": "Router",
@@ -40,6 +41,7 @@ export default {
"status.online": "online",
"status.offline": "offline",
"status.checking": "Prüfe…",
"status.noKey": "kein Key",
"theme.toggle": "Design",
"logout": "Abmelden",
@@ -88,6 +90,7 @@ export default {
"dash.health.title": "Status",
"dash.health.reachable": "erreichbar",
"dash.health.unreachable": "nicht erreichbar",
"dash.health.getKey": "Router-Key holen",
// models
"models.title": "Modelle",
@@ -225,6 +228,18 @@ export default {
"settings.router.timeout": "Timeout (ms)",
"settings.router.verifyTls": "TLS-Zertifikat prüfen",
"settings.router.taxonomy": "Taxonomie-ID",
"settings.router.getKey": "Router-Key holen oder aufladen:",
"settings.router.secret": "Router-Key (Wert)",
"settings.router.test": "Router testen",
"router.probe.ok": "Router OK ({ms} ms)",
"router.probe.noKey": "Router-Key ist nicht gesetzt",
"router.probe.auth": "Key abgelehnt — ungültig oder widerrufen",
"router.probe.payment": "Zahlung erforderlich — Tarif abgelaufen oder Kontingent erschöpft",
"router.probe.quota": "Ratenlimit überschritten",
"router.probe.timeout": "Router-Zeitüberschreitung",
"router.probe.unreachable": "Router nicht erreichbar",
"router.probe.error": "Router-Fehler",
"router.usage": "Verbrauch",
"settings.route": "Routing-Eingabe",
"settings.route.strategy": "Textstrategie",
"settings.route.maxChars": "Max. Zeichen",
+39
View File
@@ -129,6 +129,8 @@ export default {
"models.form.secretMissing": "not set",
"models.form.secretPlaceholder": "set or replace key (write-only)",
"models.deleteConfirm": "Delete model \"{id}\"?",
"models.managed": "managed",
"models.managedHint": "Provided by the managed CMF server (Settings → Local models). Configure it there.",
// routing
"routing.title": "Routing",
@@ -242,6 +244,43 @@ export default {
"settings.cmf.threads": "Worker threads",
"settings.cmf.gpu": "Use GPU (Metal / wgpu)",
"settings.cmf.speedNote": "Threads parallelise the decode matvecs (~3x faster; 0 = runtime default). GPU offload mainly helps prefill of long prompts — decode is memory-bandwidth-bound, so on Apple Silicon it is roughly neutral.",
"settings.cmf.pick": "select a model",
"settings.cmf.models": "Local models",
"settings.cmf.modelsHelp": "Each row is one model the gateway runs (its own cortiq serve on its own port) and registers in the pool. Add as many as you have .cmf files and free ports.",
"settings.cmf.rowHint": "id · .cmf file · port · threads · GPU. Changes apply on restart; deleting a row stops that server.",
"settings.cmf.addServer": "+ Add local model",
"settings.cmf.checkPort": "Check port",
"settings.cmf.portFree": "port is free",
"settings.cmf.portBusy": "port is in use",
"settings.cmf.useSuggested": "use {port}",
"settings.cmf.serving": "serving {model} on :{port}",
"settings.cmf.intro": "Run models locally with the cortiq engine — works even offline or when the router is down.",
"settings.cmf.runtime": "cortiq runtime",
"settings.cmf.runtimeHelp": "The `cortiq` command-line tool that loads and runs .cmf models. Install it once before starting a local server.",
"settings.cmf.install": "Install cortiq",
"settings.cmf.reinstall": "Reinstall / update",
"settings.cmf.installing": "installing… (compiles, a few minutes)",
"settings.cmf.installedOk": "cortiq installed",
"settings.cmf.notInstalled": "not installed",
"settings.cmf.addModel": "Add a model",
"settings.cmf.addModelHelp": "Models are downloaded and converted in the Import tab (HuggingFace → .cmf). Then point the local server at the .cmf file below.",
"settings.cmf.importNote": "No model yet? Import one:",
"settings.cmf.importLink": "Open Import",
"settings.cmf.modelHelp": "Path to the .cmf file to serve (e.g. models/qwen.cmf), produced by the Import tab. Running several models for different tasks? Import them and switch this path.",
"settings.cmf.portHelp": "TCP port the local model server listens on (default 8081). Change only if it clashes with another service.",
"settings.cmf.threadsHelp": "How many CPU cores to use for generation. More = faster (~3× vs 1 core); 0 lets the runtime decide.",
"settings.cmf.gpuHelp": "Offload matrix math to the GPU (Metal/wgpu). On Apple Silicon this mostly speeds up long prompts — decode is memory-bandwidth-bound.",
"settings.cmf.localOnlyHelp": "Route every request to the local model, ignoring the router and any cloud providers.",
"settings.cmf.manageHelp": "The gateway starts `cortiq serve` for the .cmf below at launch and routes to it. Needs cortiq installed (step 1). Applies on restart.",
"settings.cmf.autoInstallHelp": "If cortiq is missing at startup, install it automatically from crates.io.",
"settings.cmf.autoUpdateHelp": "On startup, upgrade cortiq to the newest release when a newer one exists.",
"settings.router.enabledHelp": "Use the cortiq-router to classify each request (task type + complexity) and pick a model. Off = use the local routing table only.",
"settings.router.verifyTlsHelp": "Verify the router's TLS certificate. Turn off only for a self-signed / IP-bound certificate.",
"settings.telemetry.metricsHelp": "Expose Prometheus metrics at GET /metrics.",
"settings.cortiq.echoHelp": "Add a `cortiq` field (task, model, cost) to each response body. Response headers are always sent.",
"settings.stats.enabledHelp": "Record recent requests to power the dashboard and charts.",
"settings.cache.enabledHelp": "Semantic cache: return a saved answer when a new prompt is very similar to a past one, skipping the model call. Needs an embedding model in the pool.",
"settings.log.bodiesHelp": "Log full request/response message bodies. Off by default for privacy.",
"settings.router.taxonomy": "Taxonomy id",
"settings.router.getKey": "Get or top up a router key at",
"settings.router.secret": "Router key (value)",
+15
View File
@@ -33,6 +33,7 @@ export default {
"nav.playground": "Playground",
"nav.settings": "Configuración",
"nav.foot": "Gateway · v{v}",
"nav.billing": "Clave del router y pagos",
// topbar / status
"status.router": "Router",
@@ -40,6 +41,7 @@ export default {
"status.online": "en línea",
"status.offline": "sin conexión",
"status.checking": "verificando…",
"status.noKey": "sin clave",
"theme.toggle": "Tema",
"logout": "Cerrar sesión",
@@ -88,6 +90,7 @@ export default {
"dash.health.title": "Estado",
"dash.health.reachable": "accesible",
"dash.health.unreachable": "inaccesible",
"dash.health.getKey": "Obtener una clave del router",
// models
"models.title": "Modelos",
@@ -225,6 +228,18 @@ export default {
"settings.router.timeout": "Tiempo de espera (ms)",
"settings.router.verifyTls": "Verificar certificado TLS",
"settings.router.taxonomy": "ID de taxonomía",
"settings.router.getKey": "Obtén o recarga una clave del router en",
"settings.router.secret": "Clave del router (valor)",
"settings.router.test": "Probar router",
"router.probe.ok": "Router OK ({ms} ms)",
"router.probe.noKey": "La clave del router no está configurada",
"router.probe.auth": "Clave rechazada — inválida o revocada",
"router.probe.payment": "Pago requerido — plan vencido o cuota agotada",
"router.probe.quota": "Límite de tasa excedido",
"router.probe.timeout": "Tiempo de espera del router agotado",
"router.probe.unreachable": "Router inaccesible",
"router.probe.error": "Error del router",
"router.usage": "Consumo",
"settings.route": "Entrada de enrutamiento",
"settings.route.strategy": "Estrategia de texto",
"settings.route.maxChars": "Máx. caracteres",
+15
View File
@@ -33,6 +33,7 @@ export default {
"nav.playground": "Playground",
"nav.settings": "Paramètres",
"nav.foot": "Gateway · v{v}",
"nav.billing": "Clé routeur & facturation",
// topbar / status
"status.router": "Routeur",
@@ -40,6 +41,7 @@ export default {
"status.online": "en ligne",
"status.offline": "hors ligne",
"status.checking": "vérification…",
"status.noKey": "pas de clé",
"theme.toggle": "Thème",
"logout": "Déconnexion",
@@ -88,6 +90,7 @@ export default {
"dash.health.title": "Santé",
"dash.health.reachable": "accessible",
"dash.health.unreachable": "inaccessible",
"dash.health.getKey": "Obtenir une clé routeur",
// models
"models.title": "Modèles",
@@ -225,6 +228,18 @@ export default {
"settings.router.timeout": "Délai d'expiration (ms)",
"settings.router.verifyTls": "Vérifier le certificat TLS",
"settings.router.taxonomy": "ID de taxonomie",
"settings.router.getKey": "Obtenir ou recharger une clé routeur sur",
"settings.router.secret": "Clé routeur (valeur)",
"settings.router.test": "Tester le routeur",
"router.probe.ok": "Routeur OK ({ms} ms)",
"router.probe.noKey": "La clé routeur n'est pas définie",
"router.probe.auth": "Clé rejetée — invalide ou révoquée",
"router.probe.payment": "Paiement requis — forfait expiré ou quota épuisé",
"router.probe.quota": "Limite de débit dépassée",
"router.probe.timeout": "Délai du routeur dépassé",
"router.probe.unreachable": "Routeur inaccessible",
"router.probe.error": "Erreur du routeur",
"router.usage": "Consommation",
"settings.route": "Entrée de routage",
"settings.route.strategy": "Stratégie de texte",
"settings.route.maxChars": "Caractères max",
+39
View File
@@ -128,6 +128,8 @@ export default {
"models.form.secretMissing": "не задан",
"models.form.secretPlaceholder": "задать или заменить ключ (только запись)",
"models.deleteConfirm": "Удалить модель \"{id}\"?",
"models.managed": "управляемая",
"models.managedHint": "Предоставляется CMF-сервером (Настройки → Локальные модели). Настраивается там.",
// routing
"routing.title": "Роутинг",
@@ -241,6 +243,43 @@ export default {
"settings.cmf.threads": "Рабочие потоки",
"settings.cmf.gpu": "Использовать GPU (Metal / wgpu)",
"settings.cmf.speedNote": "Потоки распараллеливают декод-матвеки (~×3; 0 = по умолчанию рантайма). GPU-оффлоад помогает в основном на prefill длинных промптов — декод упирается в пропускную способность памяти, поэтому на Apple Silicon почти нейтрален.",
"settings.cmf.pick": "выберите модель",
"settings.cmf.models": "Локальные модели",
"settings.cmf.modelsHelp": "Каждая строка — отдельная модель: шлюз поднимает свой cortiq serve на своём порту и регистрирует её в пуле. Добавляйте столько, сколько есть .cmf-файлов и свободных портов.",
"settings.cmf.rowHint": "id · файл .cmf · порт · потоки · GPU. Изменения применяются после перезапуска; удаление строки останавливает сервер.",
"settings.cmf.addServer": "+ Добавить модель",
"settings.cmf.checkPort": "Проверить порт",
"settings.cmf.portFree": "порт свободен",
"settings.cmf.portBusy": "порт занят",
"settings.cmf.useSuggested": "занять {port}",
"settings.cmf.serving": "работает: {model} на :{port}",
"settings.cmf.intro": "Запускайте модели локально движком cortiq — работает даже офлайн или когда роутер недоступен.",
"settings.cmf.runtime": "Среда cortiq",
"settings.cmf.runtimeHelp": "Инструмент командной строки `cortiq`, который загружает и запускает .cmf-модели. Установите один раз перед запуском локального сервера.",
"settings.cmf.install": "Установить cortiq",
"settings.cmf.reinstall": "Переустановить / обновить",
"settings.cmf.installing": "устанавливается… (компилируется, пара минут)",
"settings.cmf.installedOk": "cortiq установлен",
"settings.cmf.notInstalled": "не установлен",
"settings.cmf.addModel": "Добавить модель",
"settings.cmf.addModelHelp": "Модели скачиваются и конвертируются во вкладке Импорт (HuggingFace → .cmf). Затем укажите путь к .cmf ниже.",
"settings.cmf.importNote": "Ещё нет модели? Импортируйте:",
"settings.cmf.importLink": "Открыть Импорт",
"settings.cmf.modelHelp": "Путь к .cmf-файлу для запуска (напр. models/qwen.cmf), создаётся во вкладке Импорт. Несколько моделей под разные задачи? Импортируйте их и меняйте этот путь.",
"settings.cmf.portHelp": "TCP-порт локального сервера моделей (по умолчанию 8081). Меняйте, только если конфликтует с другим сервисом.",
"settings.cmf.threadsHelp": "Сколько ядер CPU использовать для генерации. Больше = быстрее (~×3 против 1 ядра); 0 — на усмотрение рантайма.",
"settings.cmf.gpuHelp": "Выгрузить матричные операции на GPU (Metal/wgpu). На Apple Silicon ускоряет в основном длинные промпты — декод упирается в пропускную способность памяти.",
"settings.cmf.localOnlyHelp": "Направлять все запросы в локальную модель, игнорируя роутер и облачные провайдеры.",
"settings.cmf.manageHelp": "Шлюз запускает `cortiq serve` для .cmf ниже при старте и маршрутизирует в него. Нужен установленный cortiq (шаг 1). Применяется после перезапуска.",
"settings.cmf.autoInstallHelp": "Если cortiq не найден при старте — установить автоматически с crates.io.",
"settings.cmf.autoUpdateHelp": "При старте обновлять cortiq до новейшей версии, когда есть новее.",
"settings.router.enabledHelp": "Использовать cortiq-router для классификации запроса (тип задачи + сложность) и выбора модели. Выкл = только локальная таблица маршрутизации.",
"settings.router.verifyTlsHelp": "Проверять TLS-сертификат роутера. Выключайте только для самоподписанного / привязанного к IP сертификата.",
"settings.telemetry.metricsHelp": "Отдавать метрики Prometheus по GET /metrics.",
"settings.cortiq.echoHelp": "Добавлять поле `cortiq` (задача, модель, стоимость) в тело каждого ответа. Заголовки ответа шлются всегда.",
"settings.stats.enabledHelp": "Записывать недавние запросы для дашборда и графиков.",
"settings.cache.enabledHelp": "Семантический кэш: возвращать сохранённый ответ, когда новый промпт очень похож на прошлый, пропуская вызов модели. Нужна embedding-модель в пуле.",
"settings.log.bodiesHelp": "Логировать полные тела запросов/ответов. По умолчанию выкл ради приватности.",
"settings.router.taxonomy": "ID таксономии",
"settings.router.getKey": "Получить или пополнить ключ роутера:",
"settings.router.secret": "Ключ роутера (значение)",
+15
View File
@@ -33,6 +33,7 @@ export default {
"nav.playground": "Oyun alanı",
"nav.settings": "Ayarlar",
"nav.foot": "Gateway · v{v}",
"nav.billing": "Router anahtarı ve ödeme",
// topbar / status
"status.router": "Yönlendirici",
@@ -40,6 +41,7 @@ export default {
"status.online": "çevrimiçi",
"status.offline": "çevrimdışı",
"status.checking": "kontrol ediliyor…",
"status.noKey": "anahtar yok",
"theme.toggle": "Tema",
"logout": "Çıkış yap",
@@ -88,6 +90,7 @@ export default {
"dash.health.title": "Sağlık",
"dash.health.reachable": "erişilebilir",
"dash.health.unreachable": "erişilemiyor",
"dash.health.getKey": "Router anahtarı al",
// models
"models.title": "Modeller",
@@ -225,6 +228,18 @@ export default {
"settings.router.timeout": "Zaman aşımı (ms)",
"settings.router.verifyTls": "TLS sertifikasını doğrula",
"settings.router.taxonomy": "Taksonomi id",
"settings.router.getKey": "Router anahtarı almak veya bakiye yüklemek için:",
"settings.router.secret": "Router anahtarı (değer)",
"settings.router.test": "Router'ı test et",
"router.probe.ok": "Router çalışıyor ({ms} ms)",
"router.probe.noKey": "Router anahtarı ayarlanmadı",
"router.probe.auth": "Anahtar reddedildi — geçersiz veya iptal edilmiş",
"router.probe.payment": "Ödeme gerekli — plan süresi doldu veya kota tükendi",
"router.probe.quota": "Hız sınırı aşıldı",
"router.probe.timeout": "Router zaman aşımına uğradı",
"router.probe.unreachable": "Router erişilemiyor",
"router.probe.error": "Router hatası",
"router.usage": "Kullanım",
"settings.route": "Yönlendirme girişi",
"settings.route.strategy": "Metin stratejisi",
"settings.route.maxChars": "Maks. karakter",
+15
View File
@@ -33,6 +33,7 @@ export default {
"nav.playground": "沙盒",
"nav.settings": "设置",
"nav.foot": "网关 · v{v}",
"nav.billing": "路由器密钥与充值",
// topbar / status
"status.router": "路由器",
@@ -40,6 +41,7 @@ export default {
"status.online": "在线",
"status.offline": "离线",
"status.checking": "检测中…",
"status.noKey": "无密钥",
"theme.toggle": "主题",
"logout": "退出",
@@ -88,6 +90,7 @@ export default {
"dash.health.title": "健康状态",
"dash.health.reachable": "可达",
"dash.health.unreachable": "不可达",
"dash.health.getKey": "获取路由器密钥",
// models
"models.title": "模型",
@@ -225,6 +228,18 @@ export default {
"settings.router.timeout": "超时(ms)",
"settings.router.verifyTls": "验证 TLS 证书",
"settings.router.taxonomy": "分类体系 ID",
"settings.router.getKey": "获取或充值路由器密钥:",
"settings.router.secret": "路由器密钥(值)",
"settings.router.test": "测试路由器",
"router.probe.ok": "路由器正常({ms} ms)",
"router.probe.noKey": "未设置路由器密钥",
"router.probe.auth": "密钥被拒绝——无效或已吊销",
"router.probe.payment": "需要付款——套餐到期或配额用尽",
"router.probe.quota": "超出速率限制",
"router.probe.timeout": "路由器响应超时",
"router.probe.unreachable": "路由器不可达",
"router.probe.error": "路由器错误",
"router.usage": "用量",
"settings.route": "路由输入",
"settings.route.strategy": "文本策略",
"settings.route.maxChars": "最大字符数",
+27 -3
View File
@@ -3,6 +3,7 @@ import { h, mount, money, num, ms, pct, timeAgo } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { lineChart, barList } from "../charts.js";
import { SITE_URL } from "../app.js";
const RANGES = ["1h", "24h", "7d"];
const GROUPS = ["model", "tier", "label", "account"];
@@ -186,7 +187,19 @@ function recentTable(rows) {
);
}
function keySourceLabel(src) {
if (src === "store") return t("models.form.secretStored");
if (src === "env") return t("models.form.secretEnv");
return t("models.form.secretMissing");
}
function healthCard(hh) {
const r = hh.router || {};
const keySrc = r.key_source || "none";
const err = r.last_error;
// problems fixed on the site (get / pay for a key) get the top-up link
const needsSite =
!r.reachable || keySrc === "missing" || (err && ["auth", "payment", "quota"].includes(err.kind));
return h(
"div",
{ class: "card" },
@@ -196,9 +209,20 @@ function healthCard(hh) {
{ class: "flex wrap", style: "margin-bottom:12px" },
h(
"span",
{ class: "badge " + (hh.router?.reachable ? "ok" : "bad"), title: hh.router?.url || "" },
t("status.router") + ": " + (hh.router?.reachable ? t("dash.health.reachable") : t("dash.health.unreachable"))
)
{ class: "badge " + (r.reachable ? "ok" : "bad"), title: r.url || "" },
t("status.router") + ": " + (r.reachable ? t("dash.health.reachable") : t("dash.health.unreachable"))
),
keySrc !== "none"
? h(
"span",
{ class: "badge " + keyBadge(keySrc), title: r.key_env || "" },
t("models.col.key") + ": " + keySourceLabel(keySrc)
)
: null,
err ? h("span", { class: "badge bad" }, t("router.probe." + (err.kind || "error"))) : null,
needsSite
? h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, t("dash.health.getKey") + " ↗")
: null
),
h(
"div",
+332
View File
@@ -0,0 +1,332 @@
// Import a HuggingFace model → convert to a local .cmf (our format) with a
// live progress view. Search HF, pick quantization + advanced params, watch
// the conversion (multiple in parallel, resumable across tab switches), then
// register the result as a local model.
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
const QUANTS = [
["Q8_2F", "import.q.q8_2f"],
["Q8_ROW", "import.q.q8_row"],
["Q4_BLOCK", "import.q.q4"],
["VBIT", "import.q.vbit"],
["F16", "import.q.f16"],
["F32", "import.q.f32"],
];
function field(label, control, hint) {
return h("label", { class: "field" }, h("span", {}, label), control,
hint ? h("div", { class: "hint" }, hint) : null);
}
function opt(v, label, sel) {
return h("option", { value: v, selected: sel ? true : null }, label || v);
}
function fmtNum(n) {
if (n == null) return "—";
if (n >= 1e6) return (n / 1e6).toFixed(1) + "M";
if (n >= 1e3) return (n / 1e3).toFixed(1) + "k";
return String(n);
}
function fmtBytes(n) {
if (!n) return "";
const g = n / 1e9;
return g >= 1 ? g.toFixed(2) + " GB" : (n / 1e6).toFixed(0) + " MB";
}
// state → localized label (falls back to the raw state)
function stateLabel(s) {
const k = "import.state." + s;
const v = t(k);
return v === k ? s : v;
}
export function renderImport() {
let selected = null;
let lastModels = []; // cache of the last search, so re-highlighting needs no refetch
let jobsTimer = null;
const searchIn = h("input", {
class: "hf-search",
placeholder: t("import.searchPlaceholder"),
autofocus: true,
});
const results = h("div", { class: "hf-grid" });
const formHost = h("div", {});
const jobsHost = h("div", {});
// ── live HF search (debounced) ──
let debounce = null;
function renderCards(models) {
if (!models.length) {
mount(results, h("div", { class: "muted pad" }, t("import.noResults")));
return;
}
mount(results, ...models.map(modelCard));
}
async function doSearch() {
const q = searchIn.value.trim();
mount(results, h("div", { class: "muted pad" }, t("import.searching")));
try {
const r = await api.hfSearch(q, 24);
lastModels = r.models || [];
renderCards(lastModels);
} catch (e) {
mount(results, h("div", { class: "err pad" }, "HF: " + e.message));
}
}
searchIn.addEventListener("input", () => {
clearTimeout(debounce);
debounce = setTimeout(doSearch, 300);
});
function modelCard(m) {
const card = h(
"div",
{ class: "hf-card" + (selected && selected.id === m.id ? " sel" : "") },
h("div", { class: "hf-id" }, m.id),
h(
"div",
{ class: "hf-meta" },
m.pipeline_tag ? h("span", { class: "badge" }, m.pipeline_tag) : null,
h("span", { class: "muted" }, "▼ " + fmtNum(m.downloads)),
h("span", { class: "muted" }, "♥ " + fmtNum(m.likes)),
(m.tags || []).some((x) => x === "gated") ? h("span", { class: "badge bad" }, "gated") : null
)
);
card.addEventListener("click", () => selectModel(m));
return card;
}
function selectModel(m) {
selected = m;
renderCards(lastModels); // re-highlight from cache — no refetch, no flash
renderForm();
formHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
}
// ── conversion form ──
function renderForm() {
if (!selected) {
mount(formHost);
return;
}
const nameIn = h("input", { value: (selected.id.split("/").pop() || "model").toLowerCase() });
const quantSel = h("select", {}, ...QUANTS.map(([v], i) => opt(v, v, i === 0)));
const quantHint = h("div", { class: "hint" }, t("import.q.q8_2f"));
quantSel.addEventListener("change", () => {
const k = QUANTS.find(([v]) => v === quantSel.value);
quantHint.textContent = k ? t(k[1]) : "";
});
// advanced (collapsed)
const linearSel = h("select", {}, opt("", t("import.adv.auto")), opt("gated_delta_net", "gated_delta_net"), opt("vmf_phase", "vmf_phase"));
const nphaseIn = h("input", { type: "number", min: "1", placeholder: "16" });
const vbitSel = h("select", {}, opt("", t("import.adv.auto")), opt("log2", "log2"), opt("cubic", "cubic"));
const meanBitsIn = h("input", { type: "number", step: "0.1", placeholder: "5.0" });
const shardIn = h("input", { type: "number", step: "0.5", placeholder: t("import.adv.noShard") });
const skipMtp = h("input", { type: "checkbox" });
const advBody = h(
"div",
{ class: "adv-body", style: "display:none" },
h("div", { class: "row" },
field(t("import.adv.linearCore"), linearSel, t("import.adv.linearHint")),
field(t("import.adv.nphase"), nphaseIn)),
h("div", { class: "row" },
field(t("import.adv.vbitShape"), vbitSel),
field(t("import.adv.meanBits"), meanBitsIn)),
h("div", { class: "row" },
field(t("import.adv.shard"), shardIn),
field("MTP", h("label", { class: "check" }, skipMtp, t("import.adv.skipMtp")))),
);
const advToggle = h("button", { class: "btn ghost sm" }, "⚙ " + t("import.adv.title"));
advToggle.addEventListener("click", () => {
advBody.style.display = advBody.style.display === "none" ? "block" : "none";
});
const goBtn = h("button", { class: "btn primary" }, "⬇ " + t("import.convert"));
goBtn.addEventListener("click", async () => {
const params = {
repo: selected.id,
quant: quantSel.value,
name: nameIn.value.trim(),
skip_mtp: skipMtp.checked,
};
if (linearSel.value) params.linear_core = linearSel.value;
if (nphaseIn.value) params.nphase = parseInt(nphaseIn.value);
if (vbitSel.value) params.vbit_shape = vbitSel.value;
if (meanBitsIn.value) params.mean_bits = parseFloat(meanBitsIn.value);
if (shardIn.value) params.shard_max_gb = parseFloat(shardIn.value);
goBtn.disabled = true;
try {
await api.startImport(params);
toast(t("import.started"), "ok");
await refreshJobs(); // picks up the new job + (re)starts polling
jobsHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
} catch (e) {
toast(e.message, "err");
}
goBtn.disabled = false;
});
mount(
formHost,
h("div", { class: "card import-form" },
h("div", { class: "card-title" }, t("import.configure") + ": " + selected.id),
h("div", { class: "row" },
field(t("import.outName"), nameIn, t("import.outNameHint")),
field(t("import.quant"), quantSel)),
quantHint,
h("div", { class: "adv" }, advToggle, advBody),
h("div", { class: "actions" }, goBtn),
)
);
}
// ── jobs panel: all conversions, live, resumable, cancellable ──
function ensurePolling() {
if (!jobsTimer) {
jobsTimer = setInterval(() => {
// stop once this view instance was replaced (navigated away)
if (!root.isConnected) {
stopPolling();
return;
}
refreshJobs();
}, 1500);
}
}
function stopPolling() {
if (jobsTimer) {
clearInterval(jobsTimer);
jobsTimer = null;
}
}
// Load + render all jobs. Runs on mount (before `root` is attached — so no
// isConnected guard here) and on each poll tick.
async function refreshJobs() {
let jobs = [];
try {
jobs = (await api.listImports()).jobs || [];
} catch {
return;
}
jobs = jobs.filter((j) => j.state !== "cancelled"); // hide user-cancelled jobs
renderJobs(jobs);
if (jobs.some((j) => j.state === "running")) ensurePolling();
else stopPolling();
}
function jobCard(j) {
const running = j.state === "running";
const pct = j.progress != null ? Math.round(j.progress * 100) : null;
// det = solid bar at a known %, run = indeterminate slider, ok/err = terminal.
let bar;
if (j.state === "done") {
bar = h("div", { class: "pbar-fill ok", style: "width:100%" });
} else if (running && pct != null) {
bar = h("div", { class: "pbar-fill det", style: `width:${pct}%` });
} else if (running) {
bar = h("div", { class: "pbar-fill run" }); // CSS drives width + animation
} else {
bar = h("div", { class: "pbar-fill err", style: "width:100%" }); // error / cancelled
}
const barWrap = h("div", { class: "pbar" }, bar);
const badge = h(
"span",
{ class: "badge " + (j.state === "done" ? "" : running ? "" : "bad") },
stateLabel(j.state) +
(running && pct != null ? " · " + pct + "%" : "") +
(j.size_bytes ? " · " + fmtBytes(j.size_bytes) : "")
);
const phase = h("div", { class: "muted small" }, j.phase || "");
const logEl = h("pre", { class: "conv-log" }, (j.log || []).slice(-10).join("\n"));
const actions = [];
if (running) {
const c = h("button", { class: "btn ghost sm" }, "✕ " + t("import.cancel"));
c.addEventListener("click", async () => {
c.disabled = true;
try {
await api.cancelImport(j.id);
toast(t("import.cancelling"), "warn");
refreshJobs();
} catch (e) {
toast(e.message, "err");
c.disabled = false;
}
});
actions.push(c);
}
if (j.state === "done") {
const r = h("button", { class: "btn primary sm" }, "✓ " + t("import.register"));
r.addEventListener("click", async () => {
r.disabled = true;
try {
const rr = await api.registerImport(j.id);
toast(t("import.registered", { id: rr.model_id }), "ok");
} catch (e) {
toast(e.message, "err");
r.disabled = false;
}
});
actions.push(r);
}
if (!running) {
// delete the job + its converted .cmf file(s) from disk
const d = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("import.delete"));
d.addEventListener("click", async () => {
if (!window.confirm(t("import.deleteConfirm"))) return;
d.disabled = true;
try {
await api.deleteImport(j.id);
toast(t("import.deleted"), "ok");
refreshJobs();
} catch (e) {
toast(e.message, "err");
d.disabled = false;
}
});
actions.push(d);
}
return h(
"div",
{ class: "job" },
h("div", { class: "job-head" }, h("b", {}, j.repo || j.id), badge),
barWrap,
phase,
logEl,
actions.length ? h("div", { class: "actions" }, ...actions) : null
);
}
function renderJobs(jobs) {
if (!jobs.length) {
mount(jobsHost);
return;
}
mount(
jobsHost,
h(
"div",
{ class: "card" },
h("div", { class: "card-title" }, t("import.jobs")),
...jobs.map(jobCard)
)
);
}
const root = h(
"div",
{},
h("div", { class: "view-head" },
h("h2", {}, t("nav.import")),
h("div", { class: "muted" }, t("import.subtitle"))),
jobsHost, // progress panel on top, above the search
h("div", { class: "card" }, searchIn, results),
formHost
);
doSearch(); // initial trending list (mounts into `results`)
refreshJobs(); // resume: show any running/recent conversions immediately
return root;
}
+7 -1
View File
@@ -2,6 +2,7 @@
import { h } from "../ui.js";
import { t } from "../i18n.js";
import { setToken } from "../api.js";
import { SITE_URL } from "../app.js";
export function renderLogin(onOk) {
const input = h("input", {
@@ -34,7 +35,12 @@ export function renderLogin(onOk) {
h("p", { class: "sub" }, t("login.subtitle")),
h("label", { class: "field" }, h("span", {}, t("login.token")), input),
h("button", { class: "btn primary", style: "width:100%", onClick: submit }, t("login.submit")),
h("p", { class: "hint", style: "margin-top:16px" }, t("login.hint"))
h("p", { class: "hint", style: "margin-top:16px" }, t("login.hint")),
h(
"p",
{ class: "hint", style: "margin-top:8px" },
h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, t("nav.billing") + " · api.allaigate.com ↗")
)
)
);
}
+3 -2
View File
@@ -168,7 +168,8 @@ function row(m, meta, reload) {
return h(
"tr",
{},
h("td", { class: "mono" }, m.id),
h("td", { class: "mono" }, m.id,
m.managed ? h("span", { class: "badge", style: "margin-left:6px", title: t("models.managedHint") }, t("models.managed")) : null),
h("td", {}, m.provider),
h("td", { class: "mono" }, m.model),
h("td", {}, h("span", { class: "badge " + (m.cost_tier || "") }, m.cost_tier || "—")),
@@ -182,7 +183,7 @@ function row(m, meta, reload) {
"div",
{ class: "flex", style: "justify-content:flex-end" },
probeBtn,
h("button", { class: "btn sm ghost", onClick: () => openModal(meta, m, reload) }, t("common.edit")),
m.managed ? null : h("button", { class: "btn sm ghost", onClick: () => openModal(meta, m, reload) }, t("common.edit")),
h(
"button",
{
+191 -57
View File
@@ -4,12 +4,22 @@ import { t } from "../i18n.js";
import { api } from "../api.js";
import { appState, SITE_URL } from "../app.js";
function field(label, control, hint) {
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
// A small "?" badge with a hover/focus tooltip explaining a setting.
function help(text) {
return text ? h("span", { class: "help", title: text, tabindex: "0", role: "img", "aria-label": text }, "?") : null;
}
function check(label, checked) {
function field(label, control, hint, helpText) {
return h(
"label",
{ class: "field" },
h("span", {}, label, helpText ? " " : null, help(helpText)),
control,
hint ? h("div", { class: "hint" }, hint) : null
);
}
function check(label, checked, helpText) {
const cb = h("input", { type: "checkbox", checked: checked ? true : null });
return { node: h("label", { class: "check" }, cb, label), cb };
return { node: h("label", { class: "check" }, cb, label, helpText ? " " : null, help(helpText)), cb };
}
// Flatten router usage (nested account/usage objects) into readable, wrapping
@@ -59,8 +69,8 @@ export async function renderSettings() {
const rKeyEnv = h("input", { value: s.router?.api_key_env || "", placeholder: "CORTIQ_ROUTER_KEY" });
const rTimeout = h("input", { type: "number", value: s.router?.timeout_ms ?? 800 });
const rTax = h("input", { value: s.router?.taxonomy_id || "" });
const rVerify = check(t("settings.router.verifyTls"), s.router?.verify_tls);
const rEnabled = check(t("settings.router.enabled"), s.router?.enabled !== false);
const rVerify = check(t("settings.router.verifyTls"), s.router?.verify_tls, t("settings.router.verifyTlsHelp"));
const rEnabled = check(t("settings.router.enabled"), s.router?.enabled !== false, t("settings.router.enabledHelp"));
// router key value (write-only; stored in the secret store, like model keys)
const rKeySource =
(secretsData.secrets || []).find((x) => x.name === (s.router?.api_key_env || "CORTIQ_ROUTER_KEY"))?.source ||
@@ -108,37 +118,16 @@ export async function renderSettings() {
rKeyBadge.className = "badge store";
return true;
}
const saveKeyBtn = h("button", { class: "btn sm" }, "✓ " + t("settings.router.saveKey"));
saveKeyBtn.addEventListener("click", async () => {
saveKeyBtn.disabled = true;
try {
if (await saveKey()) toast(t("settings.router.keySaved"), "good");
else toast(t("settings.router.keyEmpty"), "warn");
} catch (e) {
toast(String(e.message || e), "bad");
}
saveKeyBtn.disabled = false;
});
// "Test router" — real /v1/route call: tells key problems apart from a down router
const probeBtn = h("button", { class: "btn" }, t("settings.router.test"));
// Shared connection test (real /v1/route call) — tells key problems apart from
// a down router. Rendered into probeOut; also used right after saving the key.
const probeOut = h("div", { class: "hint", style: "margin-top:8px" });
probeBtn.addEventListener("click", async () => {
probeBtn.disabled = true;
async function runProbe() {
mount(probeOut, h("span", { class: "spinner" }));
try {
// If a key was just pasted but not saved, persist it first so the test uses it.
if (rSecret.value.trim()) {
try {
await saveKey();
} catch (_) {
/* fall through — probe will report no_key/auth */
}
}
const r = await api.probeRouter();
const parts = [t(probeMsgKey(r.status), { ms: r.latency_ms ?? "—" })];
if (r.message) parts.push(" — " + r.message);
const line = h("span", {}, ...parts);
const out = [line];
const out = [h("span", {}, ...parts)];
if (PROBE_SITE_STATUSES.includes(r.status)) {
out.push(" · ", h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, t("dash.health.getKey") + " ↗"));
}
@@ -150,9 +139,37 @@ export async function renderSettings() {
}
mount(probeOut, h("div", {}, h("span", { class: "badge " + (r.ok ? "ok" : "bad") }, r.ok ? "OK" : "×"), " ", ...out));
toast(t(probeMsgKey(r.status), { ms: r.latency_ms ?? "—" }), r.ok ? "good" : "bad");
return r.ok;
} catch (e) {
mount(probeOut, String(e.message || e));
return false;
}
}
const saveKeyBtn = h("button", { class: "btn sm" }, "✓ " + t("settings.router.saveKey"));
saveKeyBtn.addEventListener("click", async () => {
saveKeyBtn.disabled = true;
try {
if (await saveKey()) {
toast(t("settings.router.keySaved"), "good");
// Immediately test the connection so the user sees reachable/error at once,
// instead of a stale "unavailable" until something re-checks.
if (rEnabled.cb.checked) await runProbe();
} else {
toast(t("settings.router.keyEmpty"), "warn");
}
} catch (e) {
toast(String(e.message || e), "bad");
}
saveKeyBtn.disabled = false;
});
const probeBtn = h("button", { class: "btn" }, t("settings.router.test"));
probeBtn.addEventListener("click", async () => {
probeBtn.disabled = true;
// If a key was pasted but not saved, persist it first so the test uses it.
if (rSecret.value.trim()) {
try { await saveKey(); } catch (_) { /* probe will report no_key/auth */ }
}
await runProbe();
probeBtn.disabled = false;
});
// route
@@ -165,18 +182,18 @@ export async function renderSettings() {
const brCool = h("input", { value: s.breaker?.cooldown || "30s" });
// log
const logSel = h("select", {}, ...["error", "warn", "info", "debug", "trace"].map((x) => h("option", { value: x, selected: x === s.log?.level }, x)));
const logBodies = check(t("settings.log.bodies"), s.log?.bodies);
const logBodies = check(t("settings.log.bodies"), s.log?.bodies, t("settings.log.bodiesHelp"));
// telemetry / cortiq
const metricsCb = check(t("settings.telemetry.metrics"), s.telemetry?.metrics);
const echoCb = check(t("settings.cortiq.echo"), s.cortiq?.echo);
const metricsCb = check(t("settings.telemetry.metrics"), s.telemetry?.metrics, t("settings.telemetry.metricsHelp"));
const echoCb = check(t("settings.cortiq.echo"), s.cortiq?.echo, t("settings.cortiq.echoHelp"));
// stats
const statsEnabled = check(t("settings.stats.enabled"), s.stats?.enabled);
const statsEnabled = check(t("settings.stats.enabled"), s.stats?.enabled, t("settings.stats.enabledHelp"));
const statsFile = h("input", { value: s.stats?.file || "" });
const statsRet = h("input", { value: s.stats?.retention || "7d" });
// cache
const modelsData = await api.listModels();
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled);
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
const cacheEmbedSel = h(
@@ -186,15 +203,117 @@ export async function renderSettings() {
...embedModels.map((id) => h("option", { value: id, selected: id === s.cache?.embed_model }, id))
);
// local models (CMF) — run a local .cmf server and/or force local-only routing
const cmfLocalOnly = check(t("settings.cmf.localOnly"), s.cmf?.local_only);
const cmfManage = check(t("settings.cmf.manage"), s.cmf?.manage_server);
const cmfAutoInstall = check(t("settings.cmf.autoInstall"), s.cmf?.auto_install);
const cmfAutoUpdate = check(t("settings.cmf.autoUpdate"), s.cmf?.auto_update);
const cmfModel = h("input", { value: s.cmf?.local_model || "", placeholder: "models/model.cmf" });
const cmfPort = h("input", { type: "number", value: s.cmf?.local_port ?? 8081 });
const cmfModelId = h("input", { value: s.cmf?.model_id || "cmf-local" });
const cmfThreads = h("input", { type: "number", min: 0, value: s.cmf?.threads ?? 8 });
const cmfGpu = check(t("settings.cmf.gpu"), s.cmf?.gpu);
const cmfLocalOnly = check(t("settings.cmf.localOnly"), s.cmf?.local_only, t("settings.cmf.localOnlyHelp"));
const cmfManage = check(t("settings.cmf.manage"), s.cmf?.manage_server, t("settings.cmf.manageHelp"));
const cmfAutoInstall = check(t("settings.cmf.autoInstall"), s.cmf?.auto_install, t("settings.cmf.autoInstallHelp"));
const cmfAutoUpdate = check(t("settings.cmf.autoUpdate"), s.cmf?.auto_update, t("settings.cmf.autoUpdateHelp"));
// ── managed local models: one supervised `cortiq serve` per row ──
const serverRows = [];
const serversWrap = h("div", {});
const legacyServers = s.cmf?.local_model
? [{ id: s.cmf.model_id || "cmf-local", model: s.cmf.local_model, port: s.cmf.local_port ?? 8081, threads: s.cmf.threads ?? 8, gpu: !!s.cmf.gpu }]
: [];
const initServers = (s.cmf?.servers && s.cmf.servers.length) ? s.cmf.servers : legacyServers;
// available .cmf files under models_dir → offer as a dropdown, not a typed path
const cmfFiles = ((await api.cmfFiles().catch(() => ({ files: [] }))).files) || [];
const fileLabel = (f) => f.name + (f.size ? ` (${(f.size / 1073741824).toFixed(f.size >= 1073741824 ? 1 : 2)} GB)` : f.is_dir ? " (dir)" : "");
function autoId(path) {
const base = (path || "").split("/").pop().replace(/\.cmf$/i, "");
return base ? "cmf-" + base.replace(/[^a-zA-Z0-9_-]/g, "-") : "";
}
function makeServerRow(sv) {
sv = sv || { id: "", model: "", port: 8090, threads: 8, gpu: false };
const idIn = h("input", { value: sv.id || "", placeholder: "cmf-id", style: "width:130px" });
// pick from existing .cmf files rather than typing a path
const modelSel = h("select", { style: "flex:2;min-width:150px" });
const seen = new Set();
modelSel.appendChild(h("option", { value: "" }, "— " + t("settings.cmf.pick") + " —"));
cmfFiles.forEach((f) => { modelSel.appendChild(h("option", { value: f.path }, fileLabel(f))); seen.add(f.path); });
if (sv.model && !seen.has(sv.model)) modelSel.appendChild(h("option", { value: sv.model }, sv.model));
modelSel.value = sv.model || "";
const portIn = h("input", { type: "number", value: sv.port ?? 8090, style: "width:82px" });
const threadsIn = h("input", { type: "number", min: 0, value: sv.threads ?? 8, style: "width:60px", title: t("settings.cmf.threadsHelp") });
const gpuIn = h("input", { type: "checkbox", checked: sv.gpu ? true : null, title: t("settings.cmf.gpuHelp") });
const checkBtn = h("button", { class: "btn small", type: "button" }, t("settings.cmf.checkPort"));
const removeBtn = h("button", { class: "btn small danger", type: "button", title: t("common.delete") }, "✕");
const st = h("span", { class: "small" });
modelSel.addEventListener("change", () => { if (!idIn.value || idIn._auto) { idIn.value = autoId(modelSel.value); idIn._auto = true; } });
idIn.addEventListener("input", () => { idIn._auto = false; });
checkBtn.addEventListener("click", async () => {
const p = parseInt(portIn.value, 10); if (!p) return;
checkBtn.disabled = true; mount(st, t("status.checking"));
try {
const r = await api.cmfPortCheck(p);
if (r.available) mount(st, h("span", { style: "color:var(--good)" }, "✓ " + t("settings.cmf.portFree")));
else {
const kids = [h("span", { style: "color:var(--bad)" }, "✗ " + t("settings.cmf.portBusy"))];
if (r.suggested) { const b = h("button", { class: "btn small", type: "button" }, t("settings.cmf.useSuggested", { port: r.suggested })); b.addEventListener("click", () => { portIn.value = r.suggested; mount(st, ""); }); kids.push(b); }
mount(st, h("span", {}, ...kids));
}
} catch (e) { mount(st, String(e.message || e)); }
checkBtn.disabled = false;
});
const row = h("div", { class: "flex wrap", style: "gap:6px;align-items:center;margin:6px 0" },
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), removeBtn, st);
const entry = { node: row, read: () => ({ id: idIn.value.trim() || autoId(modelSel.value), model: modelSel.value.trim(), port: parseInt(portIn.value) || 8090, threads: parseInt(threadsIn.value) || 0, gpu: gpuIn.checked }) };
removeBtn.addEventListener("click", () => { const i = serverRows.indexOf(entry); if (i >= 0) serverRows.splice(i, 1); row.remove(); });
serverRows.push(entry);
serversWrap.appendChild(row);
return entry;
}
initServers.forEach((sv) => makeServerRow(sv));
const addServerBtn = h("button", { class: "btn small", type: "button" }, t("settings.cmf.addServer"));
addServerBtn.addEventListener("click", () => makeServerRow());
// "Install cortiq" — the CLI that RUNS the .cmf format. Must be present before
// a local model server can start; this is step 1 for a new user.
const cmfStatusEl = h("span", { class: "muted" }, "…");
const cmfInstallBtn = h("button", { class: "btn" }, t("settings.cmf.install"));
// live "serving <model> on :<port>" indicator for the managed local server
const cmfServingEl = h("div", { class: "hint" });
async function refreshCmfStatus() {
try {
const st = await api.cmfStatus();
cmfInstallBtn.textContent = st.installed ? t("settings.cmf.reinstall") : t("settings.cmf.install");
mount(
cmfStatusEl,
st.installed
? h("span", {}, h("span", { class: "badge ok" }, "✓"), " cortiq " + (st.version || ""))
: h("span", {}, h("span", { class: "badge bad" }, "×"), " " + t("settings.cmf.notInstalled"))
);
// per-server serving indicators
const servers = st.servers || [];
if (!servers.length) {
mount(cmfServingEl, "");
} else {
mount(cmfServingEl, h("div", {}, ...servers.map((sv) => {
if (sv.last_error) return h("div", { style: "color:var(--bad)" }, "✗ " + sv.id + ": " + sv.last_error);
if (sv.running) return h("div", { style: "color:var(--good)" },
"▶ " + t("settings.cmf.serving", { model: sv.id, port: sv.port }) + (sv.healthy ? "" : " …"));
return h("div", { class: "muted" }, "• " + sv.id + " :" + sv.port);
})));
}
} catch (_) {
mount(cmfStatusEl, "—");
}
}
cmfInstallBtn.addEventListener("click", async () => {
cmfInstallBtn.disabled = true;
mount(cmfStatusEl, h("span", {}, h("span", { class: "spinner" }), " " + t("settings.cmf.installing")));
try {
await api.cmfInstall();
// install compiles from crates.io — poll status for a few minutes
for (let i = 0; i < 120; i++) {
await new Promise((r) => setTimeout(r, 3000));
const st = await api.cmfStatus();
if (st.installed) { toast(t("settings.cmf.installedOk"), "good"); break; }
}
} catch (e) {
toast(String(e.message || e), "bad");
}
await refreshCmfStatus();
cmfInstallBtn.disabled = false;
});
refreshCmfStatus();
const saveBtn = h("button", { class: "btn primary" }, t("common.save"));
saveBtn.addEventListener("click", async () => {
@@ -240,11 +359,9 @@ export async function renderSettings() {
manage_server: cmfManage.cb.checked,
auto_install: cmfAutoInstall.cb.checked,
auto_update: cmfAutoUpdate.cb.checked,
local_model: cmfModel.value.trim(),
local_port: parseInt(cmfPort.value) || 8081,
model_id: cmfModelId.value.trim() || "cmf-local",
threads: parseInt(cmfThreads.value) || 0,
gpu: cmfGpu.cb.checked,
// the managed-models list is the source of truth; clear the legacy single
servers: serverRows.map((r) => r.read()).filter((sv) => sv.model && sv.id),
local_model: "",
},
};
// serde: drop null optionals
@@ -372,14 +489,31 @@ export async function renderSettings() {
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.cmf"))),
cmfLocalOnly.node,
cmfManage.node,
h("div", { class: "hint" }, t("settings.cmf.intro")),
// Step 1 — install the cortiq runtime (the CLI that runs .cmf models).
h(
"label",
{ class: "field" },
h("span", {}, "1. " + t("settings.cmf.runtime"), " ", help(t("settings.cmf.runtimeHelp"))),
h("div", { class: "flex", style: "gap:8px;align-items:center;flex-wrap:wrap" }, cmfInstallBtn, cmfStatusEl)
),
h("div", { class: "row" }, cmfAutoInstall.node, cmfAutoUpdate.node),
field(t("settings.cmf.model"), cmfModel),
h("div", { class: "row" }, field(t("settings.cmf.port"), cmfPort), field(t("settings.cmf.modelId"), cmfModelId)),
h("div", { class: "row" }, field(t("settings.cmf.threads"), cmfThreads), cmfGpu.node),
h("div", { class: "hint" }, t("settings.cmf.note")),
h("div", { class: "hint" }, t("settings.cmf.speedNote"))
h("div", { class: "divider" }),
// Step 2 — managed local models: one supervised cortiq serve per row, each
// on its own port. Add as many as you have .cmf files + free ports.
h("div", { class: "field" },
h("span", {}, "2. " + t("settings.cmf.models"), " ", help(t("settings.cmf.modelsHelp"))),
h("div", { class: "hint" }, t("settings.cmf.rowHint")),
h("div", { class: "hint" }, t("settings.cmf.importNote"),
" ", h("a", { href: "#/import" }, t("settings.cmf.importLink") + " ↗"))),
serversWrap,
addServerBtn,
cmfServingEl,
h("div", { class: "hint" }, t("settings.cmf.speedNote")),
h("div", { class: "divider" }),
// Step 3 — how the gateway uses them.
cmfManage.node,
cmfLocalOnly.node
)
),
h(