Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
851cc7f1cb | ||
|
|
d93ac537c8 | ||
|
|
4220bc27c2 | ||
|
|
6fdbe0e159 | ||
|
|
243d8f4c7d | ||
|
|
4f72711a1e | ||
|
|
7855472499 | ||
|
|
a6ed974ece | ||
|
|
4c11b97197 | ||
|
|
87f00ee502 | ||
|
|
4fa1b8ea22 | ||
|
|
357441e0da | ||
|
|
9d2181e82b | ||
|
|
30378f7b6a | ||
|
|
b4f78bd192 | ||
|
|
3c90dad9b9 | ||
|
|
25f93d1a27 | ||
|
|
0f89cdaa07 | ||
|
|
d211eb6150 | ||
|
|
04580196ad | ||
|
|
7a3d6a7bb4 | ||
|
|
07d011263d | ||
|
|
0040efc948 | ||
|
|
16a3e209b6 | ||
|
|
c19edee7bb | ||
|
|
7af90a5eef | ||
|
|
8d7e5fe653 | ||
|
|
2605eccb64 | ||
|
|
1317b8024b | ||
|
|
9242806795 | ||
|
|
6714c4f556 | ||
|
|
a5966dd479 | ||
|
|
e015555dff | ||
|
|
f00bf8355a | ||
|
|
ddca3a8363 | ||
|
|
a67564bc24 | ||
|
|
7dbc39c6a6 | ||
|
|
b6fdf23aa6 | ||
|
|
220df29888 | ||
|
|
74d0295181 | ||
|
|
73a2bd7b42 | ||
|
|
e97fcddf87 | ||
|
|
51487e6329 |
@@ -3,60 +3,8 @@
|
||||
All notable changes to this project are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [0.2.32] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Responsive mobile admin console**: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
|
||||
- **Accessible mobile interactions**: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.
|
||||
|
||||
### Fixed
|
||||
- **Add model dialog**: Restored the missing `opt` helper import that prevented the Add model button from opening its form.
|
||||
- **Local model settings on phones**: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.
|
||||
|
||||
## [0.2.31] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Structured O(1) Nyström Attention & Reasoning Control**: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (`o1_m`, `o1_window`, `o1_sink`), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to `think_budget = 0` for Qwen3/3.5 models), and Runtime (`skip_mtp` via `CMF_MTP=0`).
|
||||
- **Persistent Admin Token Configuration**: Support for explicit `token` field in `config/gateway.toml` under `[admin]` block to prevent authorization invalidation across restarts.
|
||||
|
||||
### Fixed
|
||||
- **JavaScript UI Helper Restoration**: Restored `opt(...)` helper in `web/js/ui.js` and `web/js/views/import.js` to ensure HF model import modal options render smoothly.
|
||||
- **Engine Command Flag Accuracy**: Fixed `CMF_MTP=0` environment flag pass-through for `cortiq serve` processes.
|
||||
|
||||
## [0.2.30] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Per-Model Generation & Reasoning Parameters**: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (`o1`), MTP (Multi-Token Prediction toggle `skip_mtp`), and default System Prompt.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Duplicate Local Model Registration**: Fixed `duplicate managed model id` conflict in `register_import` by cleanly routing managed CMF models directly into `[[cmf.servers]]` with automatic free port allocation without pushing duplicate entries to `[[models]]`.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Quantization Badge & Title for Pre-Converted Models**: Automatically detect quantization (`Q1`, `Q1T`) for ready-to-run `.cmf` models (e.g. `infosave/Bonsai-27Bcmf`) from repo name and downloaded filename, resolving default `Q8_2F` badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Automatic CMF Local Server Lifecycle**: Registering an imported `.cmf` model automatically adds it to `[[cmf.servers]]` and reloads the runtime (`cmf_runtime::manage`), automatically spawning `cortiq serve` on port 8090 so chat completion requests route seamlessly without `upstream unavailable` errors.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Multi-GB Ready CMF Download Stream**: Replaced HTTP stream decoder with a streaming process downloader (`curl`) for pre-converted `.cmf` models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
|
||||
- **Ready CMF Quantization & UI Labels**: Set correct quantization badges (`Q1`, `Q1T`) for featured ready-to-run `.cmf` models and removed misleading "converting" log messages for direct downloads.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Direct Ready CMF Download**: Importing pre-converted `.cmf` model repositories (e.g. `infosave/Bonsai-8B_2bit_cmf`) directly streams the ready `.cmf` file from Hugging Face instead of attempting a safetensors conversion without `config.json`.
|
||||
|
||||
|
||||
### Added
|
||||
- **New Quantization Schemes**: Added support for `Q4_TILED` (q4t), `Q1T` (1.58-bit ternary), `Q1S` (1-bit sparse), `Q1P` (1-bit PTQ error diffusion), and `Q1` (pure binary) in converter backend and web admin UI.
|
||||
- **Featured Ready-to-Run .cmf Models**: Added 1-tap download cards for pre-converted `.cmf` models (`infosave/Bonsai-1.7Bcmf`, `infosave/Bonsai-8B_2bit_cmf`, `infosave/Bonsai-27Bcmf`).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
|
||||
### Added
|
||||
- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
|
||||
gateway + a local Ollama model in one command. GHCR badge in the READMEs.
|
||||
|
||||
Generated
+1
-1
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
||||
|
||||
[[package]]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.33"
|
||||
version = "0.2.19"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.34"
|
||||
version = "0.2.19"
|
||||
edition = "2021"
|
||||
rust-version = "1.75"
|
||||
license = "Apache-2.0"
|
||||
|
||||
+108
-164
@@ -467,111 +467,16 @@ async fn provider_models(
|
||||
}
|
||||
// OpenAI + Anthropic both return { "data": [ { "id": ... } ] }.
|
||||
let v: Value = serde_json::from_str(&body).unwrap_or(json!({}));
|
||||
let mut models: Vec<Value> = v["data"]
|
||||
let mut models: Vec<String> = v["data"]
|
||||
.as_array()
|
||||
.map(|arr| {
|
||||
arr.iter()
|
||||
.filter_map(|m| {
|
||||
let id = m["id"].as_str()?.to_string();
|
||||
let mut obj = json!({ "id": id });
|
||||
let mut caps = vec![];
|
||||
if let Some(pricing) = m.get("pricing").filter(|v| !v.is_null()) {
|
||||
if let Some(prompt) = pricing
|
||||
.get("prompt")
|
||||
.and_then(|s| s.as_str())
|
||||
.and_then(|s| s.parse::<f64>().ok())
|
||||
{
|
||||
obj["price_in"] = json!(prompt * 1_000_000.0);
|
||||
}
|
||||
if let Some(comp) = pricing
|
||||
.get("completion")
|
||||
.and_then(|s| s.as_str())
|
||||
.and_then(|s| s.parse::<f64>().ok())
|
||||
{
|
||||
obj["price_out"] = json!(comp * 1_000_000.0);
|
||||
}
|
||||
if pricing
|
||||
.get("input_cache_read")
|
||||
.filter(|v| !v.is_null())
|
||||
.is_some()
|
||||
|| pricing
|
||||
.get("input_cache_write")
|
||||
.filter(|v| !v.is_null())
|
||||
.is_some()
|
||||
{
|
||||
caps.push("caching");
|
||||
}
|
||||
}
|
||||
if let Some(params) = m.get("supported_parameters").and_then(|p| p.as_array()) {
|
||||
let has_tools = params.iter().any(|p| {
|
||||
p.as_str() == Some("tools") || p.as_str() == Some("tool_choice")
|
||||
});
|
||||
if has_tools {
|
||||
caps.push("tools");
|
||||
}
|
||||
let has_json = params.iter().any(|p| p.as_str() == Some("response_format"));
|
||||
if has_json {
|
||||
caps.push("json");
|
||||
}
|
||||
let has_reasoning = params.iter().any(|p| {
|
||||
p.as_str() == Some("reasoning")
|
||||
|| p.as_str() == Some("include_reasoning")
|
||||
});
|
||||
if has_reasoning || m.get("reasoning").filter(|v| !v.is_null()).is_some() {
|
||||
caps.push("reasoning");
|
||||
}
|
||||
}
|
||||
if let Some(arch) = m.get("architecture").filter(|v| !v.is_null()) {
|
||||
if let Some(inputs) =
|
||||
arch.get("input_modalities").and_then(|i| i.as_array())
|
||||
{
|
||||
if inputs.iter().any(|i| i.as_str() == Some("image")) {
|
||||
caps.push("vision");
|
||||
}
|
||||
if inputs.iter().any(|i| i.as_str() == Some("audio")) {
|
||||
caps.push("audio");
|
||||
}
|
||||
}
|
||||
}
|
||||
if !caps.is_empty() {
|
||||
obj["caps"] = json!(caps);
|
||||
}
|
||||
Some(obj)
|
||||
})
|
||||
.filter_map(|m| m["id"].as_str().map(|s| s.to_string()))
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
models.sort_by(|a, b| a["id"].as_str().cmp(&b["id"].as_str()));
|
||||
|
||||
let mut resp = json!({ "ok": true, "count": models.len(), "models": models });
|
||||
|
||||
if b.provider == "openrouter" {
|
||||
if let Some(k) = key {
|
||||
if let Ok(cred_resp) = client
|
||||
.get("https://openrouter.ai/api/v1/credits")
|
||||
.header("authorization", format!("Bearer {k}"))
|
||||
.send()
|
||||
.await
|
||||
{
|
||||
if let Ok(cred_body) = cred_resp.text().await {
|
||||
let cred_v: Value = serde_json::from_str(&cred_body).unwrap_or(json!({}));
|
||||
if let Some(data) = cred_v.get("data") {
|
||||
let total = data
|
||||
.get("total_credits")
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap_or(0.0);
|
||||
let usage = data
|
||||
.get("total_usage")
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap_or(0.0);
|
||||
resp["balance"] = json!(total - usage);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ok(resp)
|
||||
models.sort();
|
||||
ok(json!({ "ok": true, "count": models.len(), "models": models }))
|
||||
}
|
||||
|
||||
async fn meta() -> ApiResult {
|
||||
@@ -701,7 +606,10 @@ fn cmf_file_for_model(
|
||||
cmf: &crate::config::CmfCfg,
|
||||
m: &crate::config::ModelCfg,
|
||||
) -> Option<std::path::PathBuf> {
|
||||
// Any loopback URL counts as local — managed servers may sit on any port.
|
||||
let local = m.base_url == cmf.cortiq_server_url
|
||||
|| m.base_url.contains("//127.0.0.1:")
|
||||
|| m.base_url.contains("//localhost:")
|
||||
|| m.base_url
|
||||
.contains(&format!("{}:{}", cmf.local_host, cmf.local_port));
|
||||
let stem = m.model.trim();
|
||||
@@ -741,12 +649,24 @@ fn cmf_path_referenced(cfg: &crate::config::Config, path: &std::path::Path) -> b
|
||||
.any(|m| cmf_file_for_model(&cfg.cmf, m).as_deref() == Some(path))
|
||||
}
|
||||
|
||||
async fn delete_model(State(state): State<SharedState>, Path(id): Path<String>) -> ApiResult {
|
||||
#[derive(Deserialize)]
|
||||
struct DeleteModelQuery {
|
||||
/// Also remove the local `.cmf` from disk. Defaults to true (the historical
|
||||
/// behaviour); the UI asks the user and passes it explicitly.
|
||||
delete_file: Option<bool>,
|
||||
}
|
||||
|
||||
async fn delete_model(
|
||||
State(state): State<SharedState>,
|
||||
Path(id): Path<String>,
|
||||
Query(q): Query<DeleteModelQuery>,
|
||||
) -> ApiResult {
|
||||
let mut cfg = current_cfg(&state);
|
||||
let delete_file = q.delete_file.unwrap_or(true);
|
||||
|
||||
// Managed local models live in [cmf], not [[models]]. Deleting one stops its
|
||||
// server, removes it from the managed config, and deletes its file — unless
|
||||
// another model still points at the same file.
|
||||
// server, removes it from the managed config, and (when asked) deletes its
|
||||
// file — unless another model still points at the same file.
|
||||
let managed = cfg.cmf.effective_servers().into_iter().find(|s| s.id == id);
|
||||
if let Some(server) = managed {
|
||||
if !cfg.models.iter().any(|x| x.id == id) {
|
||||
@@ -761,10 +681,10 @@ async fn delete_model(State(state): State<SharedState>, Path(id): Path<String>)
|
||||
let shared = cmf_path_referenced(&cfg, &path);
|
||||
state.cmf.stop_one(&id).await;
|
||||
state.reload(cfg)?;
|
||||
let file_removed = if shared {
|
||||
false
|
||||
} else {
|
||||
let file_removed = if delete_file && !shared {
|
||||
remove_cmf_path(&path)
|
||||
} else {
|
||||
false
|
||||
};
|
||||
return ok(json!({ "ok": true, "file_removed": file_removed }));
|
||||
}
|
||||
@@ -780,8 +700,8 @@ async fn delete_model(State(state): State<SharedState>, Path(id): Path<String>)
|
||||
let file = victim
|
||||
.as_ref()
|
||||
.and_then(|m| cmf_file_for_model(&cfg.cmf, m));
|
||||
// Only remove the file if no remaining model still references it.
|
||||
let file = file.filter(|p| !cmf_path_referenced(&cfg, p));
|
||||
// Only remove the file if asked to AND no remaining model still references it.
|
||||
let file = file.filter(|p| delete_file && !cmf_path_referenced(&cfg, p));
|
||||
state.reload(cfg)?;
|
||||
let file_removed = file.map(|p| remove_cmf_path(&p)).unwrap_or(false);
|
||||
ok(json!({ "ok": true, "file_removed": file_removed }))
|
||||
@@ -1000,12 +920,75 @@ async fn put_settings(State(state): State<SharedState>, Json(b): Json<SettingsBo
|
||||
{
|
||||
needs_restart = true;
|
||||
}
|
||||
let old_servers = cfg.cmf.effective_servers();
|
||||
cfg.cmf = v;
|
||||
// Replacing or renaming a managed model must not brick the save: routing
|
||||
// may still reference the old id. Map a vanished id to its replacement
|
||||
// (the new server on the same port), strip ids that are truly gone, and
|
||||
// repair the default so validate() passes.
|
||||
heal_routing_after_cmf_change(&mut cfg, &old_servers);
|
||||
}
|
||||
state.reload(cfg)?;
|
||||
ok(json!({ "ok": true, "needs_restart": needs_restart }))
|
||||
}
|
||||
|
||||
/// After the managed-servers list changed, rewrite [routing] so it only
|
||||
/// references ids that still exist. A removed server whose port is now held by
|
||||
/// a NEW server is treated as a replacement (old id → new id); other vanished
|
||||
/// ids are stripped. An emptied default falls back to the first managed server,
|
||||
/// then the first pool model.
|
||||
fn heal_routing_after_cmf_change(cfg: &mut Config, old_servers: &[crate::config::CmfServer]) {
|
||||
let servers = cfg.cmf.effective_servers();
|
||||
let known: std::collections::HashSet<String> = cfg
|
||||
.models
|
||||
.iter()
|
||||
.map(|m| m.id.clone())
|
||||
.chain(servers.iter().map(|s| s.id.clone()))
|
||||
.collect();
|
||||
let new_by_port: HashMap<u16, String> =
|
||||
servers.iter().map(|s| (s.port, s.id.clone())).collect();
|
||||
let mut rename: HashMap<String, String> = HashMap::new();
|
||||
for old in old_servers {
|
||||
if !known.contains(&old.id) {
|
||||
if let Some(new_id) = new_by_port.get(&old.port) {
|
||||
rename.insert(old.id.clone(), new_id.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
let fix = |id: &str| -> Option<String> {
|
||||
if known.contains(id) {
|
||||
Some(id.to_string())
|
||||
} else {
|
||||
rename.get(id).cloned()
|
||||
}
|
||||
};
|
||||
if !cfg.routing.default.is_empty() {
|
||||
cfg.routing.default = fix(&cfg.routing.default).unwrap_or_default();
|
||||
}
|
||||
if cfg.routing.default.is_empty() {
|
||||
cfg.routing.default = servers
|
||||
.first()
|
||||
.map(|s| s.id.clone())
|
||||
.or_else(|| cfg.models.first().map(|m| m.id.clone()))
|
||||
.unwrap_or_default();
|
||||
}
|
||||
for tt in cfg.routing.tiers.values_mut() {
|
||||
let list = match tt {
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
};
|
||||
let mut healed: Vec<String> = Vec::new();
|
||||
for id in &list {
|
||||
if let Some(new_id) = fix(id) {
|
||||
if !healed.contains(&new_id) {
|
||||
healed.push(new_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
*tt = TierTargets::List(healed);
|
||||
}
|
||||
}
|
||||
|
||||
// ───────────────────────────── cmf runtime ─────────────────────────────────
|
||||
/// Status of the `cortiq` CLI (the CMF format runtime): is it installed, which
|
||||
/// version, and whether a managed server is running. Drives the "Install cortiq"
|
||||
@@ -1026,6 +1009,7 @@ async fn cmf_status(State(state): State<SharedState>) -> ApiResult {
|
||||
"port": s.port,
|
||||
"threads": s.threads,
|
||||
"gpu": s.gpu,
|
||||
"o1": s.o1,
|
||||
"running": live.map(|l| l.running).unwrap_or(false),
|
||||
"healthy": live.map(|l| l.healthy).unwrap_or(false),
|
||||
"last_error": live.and_then(|l| l.last_error.clone()),
|
||||
@@ -1123,6 +1107,9 @@ async fn cmf_port_check(State(state): State<SharedState>, Query(q): Query<PortQu
|
||||
ok(json!({
|
||||
"port": port,
|
||||
"available": available,
|
||||
// true = held by this gateway's own managed server (a replaced/edited
|
||||
// row frees it on restart) — the UI shows this as info, not an error
|
||||
"managed": running_here,
|
||||
"detail": detail,
|
||||
"suggested": suggested,
|
||||
}))
|
||||
@@ -1399,64 +1386,21 @@ async fn register_import(State(state): State<SharedState>, Path(job): Path<Strin
|
||||
.unwrap_or_else(|| "cmf-local".into());
|
||||
let id = format!("cmf-{base}");
|
||||
let mut cfg = current_cfg(&state);
|
||||
|
||||
let is_in_models = cfg.models.iter().any(|m| m.id == id);
|
||||
let is_in_servers = cfg
|
||||
.cmf
|
||||
.servers
|
||||
.iter()
|
||||
.any(|s| s.id == id || s.model == j.output);
|
||||
|
||||
if is_in_models || is_in_servers {
|
||||
return Err(ApiError::bad(format!("model '{id}' is already registered")));
|
||||
}
|
||||
|
||||
if cfg.cmf.manage_server {
|
||||
let host = if cfg.cmf.local_host.trim().is_empty() {
|
||||
"127.0.0.1"
|
||||
} else {
|
||||
&cfg.cmf.local_host
|
||||
};
|
||||
let base_port = cfg.cmf.servers.iter().map(|s| s.port).max().unwrap_or(8089);
|
||||
let port = next_free_port(host, base_port).unwrap_or_else(|| base_port + 1);
|
||||
cfg.cmf.servers.push(crate::config::CmfServer {
|
||||
id: id.clone(),
|
||||
model: j.output.clone(),
|
||||
port,
|
||||
threads: cfg.cmf.threads,
|
||||
gpu: cfg.cmf.gpu,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
o1_m: None,
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
});
|
||||
} else {
|
||||
cfg.models.push(crate::config::ModelCfg {
|
||||
id: id.clone(),
|
||||
provider: "openai".into(),
|
||||
base_url: cfg.cmf.cortiq_server_url.clone(),
|
||||
model: base,
|
||||
cost_tier: "local".into(),
|
||||
price_in: 0.0,
|
||||
price_out: 0.0,
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
skip_mtp: false,
|
||||
});
|
||||
if cfg.models.iter().any(|m| m.id == id) {
|
||||
return Err(ApiError::bad(format!("model '{id}' already registered")));
|
||||
}
|
||||
cfg.models.push(crate::config::ModelCfg {
|
||||
id: id.clone(),
|
||||
provider: "openai".into(),
|
||||
base_url: cfg.cmf.cortiq_server_url.clone(),
|
||||
model: base,
|
||||
cost_tier: "local".into(),
|
||||
price_in: 0.0,
|
||||
price_out: 0.0,
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
});
|
||||
state.reload(cfg)?;
|
||||
ok(json!({ "ok": true, "model_id": id }))
|
||||
}
|
||||
|
||||
+6
-21
@@ -290,30 +290,15 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
||||
if server.threads > 1 {
|
||||
cmd.env("CMF_THREADS", server.threads.to_string());
|
||||
}
|
||||
// Offload large matvecs to the GPU (Metal on macOS). Memory-bandwidth-bound
|
||||
// on unified memory, so mostly a prefill win; opt-in.
|
||||
if server.gpu {
|
||||
cmd.env("CMF_GPU", "1");
|
||||
}
|
||||
// O(1) Nyström attention: layers spec + sub-parameters.
|
||||
if let Some(o1) = &server.o1 {
|
||||
let s = o1.trim();
|
||||
if !s.is_empty() {
|
||||
// Pass even "off" so it explicitly overrides a file converter hint.
|
||||
cmd.arg("--o1").arg(s);
|
||||
}
|
||||
}
|
||||
if let Some(m) = server.o1_m {
|
||||
cmd.arg("--o1-m").arg(m.to_string());
|
||||
}
|
||||
if let Some(w) = server.o1_window {
|
||||
cmd.arg("--o1-window").arg(w.to_string());
|
||||
}
|
||||
if let Some(sink) = server.o1_sink {
|
||||
cmd.arg("--o1-sink").arg(sink.to_string());
|
||||
}
|
||||
// MTP (Multi-Token Prediction) speculative decoding: the engine reads
|
||||
// CMF_MTP env; "0" disables. There is no CLI flag for this.
|
||||
if server.skip_mtp {
|
||||
cmd.env("CMF_MTP", "0");
|
||||
// Serve-time O(1) attention override: "off" forces exact attention (undoes a
|
||||
// too-aggressive --o1 conversion hint without re-converting the file).
|
||||
if !server.o1.trim().is_empty() {
|
||||
cmd.env("CMF_O1", server.o1.trim());
|
||||
}
|
||||
cmd.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
|
||||
+8
-65
@@ -83,36 +83,12 @@ pub struct CmfServer {
|
||||
/// Offload matvecs to the GPU (CMF_GPU=1).
|
||||
#[serde(default)]
|
||||
pub gpu: bool,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) Nyström attention layer spec (all | deepN | i,j,k | off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// O(1) landmark budget (default 32; ≥4).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_m: Option<u32>,
|
||||
/// O(1) exact-window width (default 128).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_window: Option<u32>,
|
||||
/// O(1) permanent exact sink keys (default 4, StreamingLLM discipline).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_sink: Option<u32>,
|
||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||
/// Serve-time O(1) attention override (exported as CMF_O1). Empty = follow
|
||||
/// the hint recorded in the .cmf; "off" = force exact attention (fixes a
|
||||
/// too-aggressive conversion without re-converting); "all"/"deepN"/"i,j,k"
|
||||
/// = force-enable on those layers.
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
pub o1: String,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -202,16 +178,7 @@ impl CmfCfg {
|
||||
port: self.local_port,
|
||||
threads: self.threads,
|
||||
gpu: self.gpu,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
o1_m: None,
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
o1: String::new(),
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
@@ -346,27 +313,6 @@ pub struct ModelCfg {
|
||||
pub api_key_env: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub caps: Vec<String>,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) constant-memory streaming attention hint (all, deep, custom, off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// Skip Multi-Token Prediction (MTP) heads.
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
}
|
||||
fn default_kind() -> String {
|
||||
"chat".into()
|
||||
@@ -522,12 +468,10 @@ pub struct AdminCfg {
|
||||
/// Whether the admin panel and admin API (`/admin`, `/admin/api/*`) are enabled.
|
||||
#[serde(default = "default_admin_enabled")]
|
||||
pub enabled: bool,
|
||||
/// Name of the env variable holding the admin token.
|
||||
/// Name of the env variable holding the admin token. If unset (or empty),
|
||||
/// a token is generated at startup and printed to the log.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token_env: Option<String>,
|
||||
/// Persistent admin token configured directly in toml.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token: Option<String>,
|
||||
/// Optional separate bind address for the admin surface (e.g. `127.0.0.1:9001`).
|
||||
/// If unset, the admin interface lives on the main `listen` address under `/admin`.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
@@ -538,7 +482,6 @@ impl Default for AdminCfg {
|
||||
Self {
|
||||
enabled: default_admin_enabled(),
|
||||
token_env: None,
|
||||
token: None,
|
||||
listen: None,
|
||||
}
|
||||
}
|
||||
|
||||
+3
-228
@@ -160,14 +160,6 @@ impl JobStore {
|
||||
}
|
||||
}
|
||||
}
|
||||
fn update_quant(&self, id: &str, quant: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.quant = quant.clone();
|
||||
if !j.log.is_empty() && j.log[0].starts_with("→ downloading ready .cmf model") {
|
||||
j.log[0] = format!("→ downloading ready .cmf model {} ({})", j.repo, quant);
|
||||
}
|
||||
}
|
||||
}
|
||||
fn set_state(&self, id: &str, state: &str) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.state = state.into();
|
||||
@@ -268,166 +260,14 @@ fn sanitize(s: &str) -> String {
|
||||
/// Map the UI quant label to the native `cortiq` quant flag.
|
||||
fn map_quant(q: &str) -> &'static str {
|
||||
match q.to_ascii_uppercase().as_str() {
|
||||
s if s.starts_with("Q4T") || s == "Q4_TILED" => "q4t",
|
||||
s if s.starts_with("Q4") => "q4",
|
||||
"Q8_2F" | "Q82F" => "q8_2f",
|
||||
"Q8_2F" => "q8_2f",
|
||||
"F16" | "FP16" => "f16",
|
||||
"VBIT" => "vbit",
|
||||
"Q1" => "q1",
|
||||
"Q1P" | "Q1_PTQ" => "q1p",
|
||||
"Q1S" | "Q1_MASK" => "q1s",
|
||||
"Q1T" | "Q1_TERNARY" => "q1t",
|
||||
_ => "q8",
|
||||
}
|
||||
}
|
||||
|
||||
fn is_cmf_repo(repo: &str) -> bool {
|
||||
let lower = repo.to_ascii_lowercase();
|
||||
lower.contains("cmf") || lower.ends_with(".cmf")
|
||||
}
|
||||
|
||||
async fn download_cmf_repo(
|
||||
repo: String,
|
||||
output_abs: String,
|
||||
hf_token: Option<String>,
|
||||
store: Arc<JobStore>,
|
||||
id: String,
|
||||
cancel: Arc<Notify>,
|
||||
) {
|
||||
store.set_progress(&id, 0.0, "listing files".into());
|
||||
|
||||
let tree_url = format!("https://huggingface.co/api/models/{repo}/tree/main?recursive=true");
|
||||
let client = reqwest::Client::new();
|
||||
let mut req = client.get(&tree_url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = &hf_token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
|
||||
let mut cmf_rel_path = None;
|
||||
if let Ok(resp) = req.send().await {
|
||||
if resp.status().is_success() {
|
||||
if let Ok(json) = resp.json::<serde_json::Value>().await {
|
||||
if let Some(arr) = json.as_array() {
|
||||
for item in arr {
|
||||
if let Some(path) = item.get("path").and_then(|p| p.as_str()) {
|
||||
if path.to_ascii_lowercase().ends_with(".cmf") {
|
||||
cmf_rel_path = Some(path.to_string());
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let cmf_path = match cmf_rel_path {
|
||||
Some(p) => p,
|
||||
None => {
|
||||
let repo_name = repo.rsplit('/').next().unwrap_or(&repo);
|
||||
format!("{repo_name}.cmf")
|
||||
}
|
||||
};
|
||||
|
||||
let path_lower = cmf_path.to_ascii_lowercase();
|
||||
let detected_quant =
|
||||
if path_lower.contains("-q1t.") || repo.to_ascii_lowercase().contains("2bit") {
|
||||
"Q1T".to_string()
|
||||
} else if path_lower.contains("-q1.") || repo.to_ascii_lowercase().contains("cmf") {
|
||||
"Q1".to_string()
|
||||
} else if path_lower.contains("-q4.") {
|
||||
"Q4".to_string()
|
||||
} else if path_lower.contains("-q8.") {
|
||||
"Q8".to_string()
|
||||
} else {
|
||||
"READY_CMF".to_string()
|
||||
};
|
||||
store.update_quant(&id, detected_quant);
|
||||
|
||||
store.push_line(&id, format!("→ downloading {cmf_path} from HuggingFace..."));
|
||||
store.set_progress(&id, 0.05, "downloading".into());
|
||||
|
||||
let download_url = format!("https://huggingface.co/{repo}/resolve/main/{cmf_path}");
|
||||
|
||||
let mut cmd = tokio::process::Command::new("curl");
|
||||
cmd.arg("-f").arg("-L").arg("-#").arg("-o").arg(&output_abs);
|
||||
|
||||
if let Some(t) = &hf_token {
|
||||
cmd.arg("-H").arg(format!("Authorization: Bearer {t}"));
|
||||
}
|
||||
|
||||
cmd.arg(&download_url);
|
||||
cmd.stderr(std::process::Stdio::piped());
|
||||
cmd.stdout(std::process::Stdio::null());
|
||||
|
||||
let mut child = match cmd.spawn() {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed to start curl download: {e}"));
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let stderr = child.stderr.take();
|
||||
let mut t_err = None;
|
||||
if let Some(err_stream) = stderr {
|
||||
let store_clone = store.clone();
|
||||
let id_clone = id.clone();
|
||||
t_err = Some(tokio::spawn(async move {
|
||||
use tokio::io::AsyncReadExt;
|
||||
let mut reader = err_stream;
|
||||
let mut buf = [0u8; 256];
|
||||
while let Ok(n) = reader.read(&mut buf).await {
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
let text = String::from_utf8_lossy(&buf[..n]);
|
||||
for word in text.split_whitespace() {
|
||||
if let Some(num_str) = word.strip_suffix('%') {
|
||||
if let Ok(pct) = num_str.parse::<f32>() {
|
||||
let frac = (pct / 100.0).clamp(0.0, 1.0);
|
||||
store_clone.set_progress(&id_clone, frac, "downloading".into());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}));
|
||||
}
|
||||
|
||||
tokio::select! {
|
||||
status_res = child.wait() => {
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
match status_res {
|
||||
Ok(status) if status.success() => {
|
||||
store.push_line(&id, "✓ done".into());
|
||||
store.finish(&id, true);
|
||||
}
|
||||
Ok(status) => {
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, format!("✗ download failed: exit code {:?}", status.code()));
|
||||
store.finish(&id, false);
|
||||
}
|
||||
Err(e) => {
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, format!("✗ download process error: {e}"));
|
||||
store.finish(&id, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
_ = cancel.notified() => {
|
||||
let _ = child.start_kill();
|
||||
let _ = child.wait().await;
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, "✗ cancelled by user — partial output removed".into());
|
||||
store.set_state(&id, "cancelled");
|
||||
}
|
||||
}
|
||||
store.drop_cancel(&id);
|
||||
}
|
||||
|
||||
/// into the job log. Fails fast if the converter script is missing.
|
||||
pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Result<String, String> {
|
||||
if p.repo.trim().is_empty() {
|
||||
@@ -486,41 +326,13 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
.unwrap_or_else(|_| output.clone());
|
||||
|
||||
let id = gen_id(&p.repo);
|
||||
let is_cmf = is_cmf_repo(&p.repo);
|
||||
let quant_display = if is_cmf {
|
||||
let r = p.repo.to_lowercase();
|
||||
if r.contains("2bit") || r.contains("q1t") {
|
||||
"Q1T".to_string()
|
||||
} else if r.contains("1.7bcmf")
|
||||
|| r.contains("27bcmf")
|
||||
|| r.contains("cmf")
|
||||
|| p.quant.is_empty()
|
||||
|| p.quant == "auto"
|
||||
|| p.quant == "Q8_2F"
|
||||
{
|
||||
"Q1".to_string()
|
||||
} else {
|
||||
p.quant.clone()
|
||||
}
|
||||
} else if p.quant.is_empty() || p.quant == "auto" {
|
||||
"q8".to_string()
|
||||
} else {
|
||||
p.quant.clone()
|
||||
};
|
||||
store.insert(Job {
|
||||
id: id.clone(),
|
||||
repo: p.repo.clone(),
|
||||
quant: quant_display.clone(),
|
||||
quant: p.quant.clone(),
|
||||
output: output_abs.clone(),
|
||||
state: "running".into(),
|
||||
log: if is_cmf {
|
||||
vec![format!(
|
||||
"→ downloading ready .cmf model {} ({})",
|
||||
p.repo, quant_display
|
||||
)]
|
||||
} else {
|
||||
vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)]
|
||||
},
|
||||
log: vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)],
|
||||
started: now(),
|
||||
finished: None,
|
||||
size_bytes: None,
|
||||
@@ -536,14 +348,6 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
std::env::var(&cfg.hf_token_env).ok()
|
||||
};
|
||||
|
||||
if is_cmf && !use_python {
|
||||
let ret_id = id.clone();
|
||||
tokio::spawn(async move {
|
||||
download_cmf_repo(p.repo, output_abs, hf_token, store, id, cancel).await;
|
||||
});
|
||||
return Ok(ret_id);
|
||||
}
|
||||
|
||||
// Program + args: `cortiq convert` by default, or the Python converter for
|
||||
// advanced options. Both stream `@PROGRESS <frac>` markers into the log.
|
||||
let (program, args, workdir): (String, Vec<String>, std::path::PathBuf) = if use_python {
|
||||
@@ -726,32 +530,3 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
|
||||
Ok(ret_id)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_map_quant_variants() {
|
||||
assert_eq!(map_quant("Q8_2F"), "q8_2f");
|
||||
assert_eq!(map_quant("q82f"), "q8_2f");
|
||||
assert_eq!(map_quant("Q8_ROW"), "q8");
|
||||
assert_eq!(map_quant("Q4_BLOCK"), "q4");
|
||||
assert_eq!(map_quant("Q4_TILED"), "q4t");
|
||||
assert_eq!(map_quant("q4t"), "q4t");
|
||||
assert_eq!(map_quant("vbit"), "vbit");
|
||||
assert_eq!(map_quant("q1"), "q1");
|
||||
assert_eq!(map_quant("Q1P"), "q1p");
|
||||
assert_eq!(map_quant("q1s"), "q1s");
|
||||
assert_eq!(map_quant("Q1T"), "q1t");
|
||||
assert_eq!(map_quant("F16"), "f16");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_cmf_repo_check() {
|
||||
assert!(is_cmf_repo("infosave/Bonsai-8B_2bit_cmf"));
|
||||
assert!(is_cmf_repo("infosave/Bonsai-1.7Bcmf"));
|
||||
assert!(is_cmf_repo("user/model.cmf"));
|
||||
assert!(!is_cmf_repo("Qwen/Qwen2.5-0.5B-Instruct"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -61,11 +61,6 @@ fn resolve_admin_token(cfg: &Config, args: &Args) -> (String, bool) {
|
||||
return (t.clone(), false);
|
||||
}
|
||||
}
|
||||
if let Some(t) = &cfg.admin.token {
|
||||
if !t.is_empty() {
|
||||
return (t.clone(), false);
|
||||
}
|
||||
}
|
||||
(admin::random_token(24), true)
|
||||
}
|
||||
|
||||
|
||||
@@ -29,7 +29,6 @@ pub struct GenParams {
|
||||
pub temperature: Option<f32>,
|
||||
pub max_tokens: Option<u32>,
|
||||
pub top_p: Option<f32>,
|
||||
pub think_budget: Option<u32>,
|
||||
pub stop: Vec<String>,
|
||||
/// Fields we do not interpret but must proxy through to the provider.
|
||||
pub passthrough: serde_json::Map<String, serde_json::Value>,
|
||||
|
||||
+2
-89
@@ -341,64 +341,7 @@ impl Pipeline {
|
||||
route_source
|
||||
);
|
||||
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat(req_call).await {
|
||||
match provider.chat(req.clone()).await {
|
||||
Ok(mut resp) => {
|
||||
let cost =
|
||||
provider.price(resp.usage.prompt_tokens, resp.usage.completion_tokens);
|
||||
@@ -472,37 +415,7 @@ impl Pipeline {
|
||||
model_id,
|
||||
route_source
|
||||
);
|
||||
// Merge per-model/server config (same as run_inner).
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat_stream(req_call).await {
|
||||
match provider.chat_stream(req.clone()).await {
|
||||
Ok(stream) => {
|
||||
let info = RouteInfo {
|
||||
task_label: decision.task_label.clone(),
|
||||
|
||||
@@ -101,7 +101,6 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
|
||||
@@ -49,10 +49,6 @@ struct OpenAiChatRequest {
|
||||
#[serde(default)]
|
||||
top_p: Option<f32>,
|
||||
#[serde(default)]
|
||||
think_budget: Option<u32>,
|
||||
#[serde(default)]
|
||||
reasoning_effort: Option<u32>,
|
||||
#[serde(default)]
|
||||
stream: bool,
|
||||
#[serde(default)]
|
||||
tools: Vec<serde_json::Value>,
|
||||
@@ -97,7 +93,6 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: req.think_budget.or(req.reasoning_effort),
|
||||
stop: Vec::new(),
|
||||
passthrough: req.rest,
|
||||
},
|
||||
|
||||
@@ -82,7 +82,6 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
|
||||
@@ -42,18 +42,9 @@ impl OpenAiProvider {
|
||||
if let Some(t) = req.params.temperature {
|
||||
body["temperature"] = t.into();
|
||||
}
|
||||
if let Some(p) = req.params.top_p {
|
||||
body["top_p"] = p.into();
|
||||
}
|
||||
if let Some(m) = req.params.max_tokens {
|
||||
body["max_tokens"] = m.into();
|
||||
}
|
||||
if let Some(tb) = req.params.think_budget {
|
||||
body["think_budget"] = tb.into();
|
||||
body["reasoning_effort"] = tb.into();
|
||||
// cortiq serve uses enable_thinking (bool): 0 = off, >0 = on
|
||||
body["enable_thinking"] = (tb > 0).into();
|
||||
}
|
||||
if !req.tools.is_empty() {
|
||||
body["tools"] = serde_json::Value::Array(req.tools.clone());
|
||||
}
|
||||
|
||||
@@ -49,13 +49,6 @@ impl Registry {
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
temperature: s.temperature,
|
||||
top_p: s.top_p,
|
||||
max_tokens: s.max_tokens,
|
||||
think_budget: s.think_budget,
|
||||
system_prompt: s.system_prompt.clone(),
|
||||
o1: s.o1.clone(),
|
||||
skip_mtp: s.skip_mtp,
|
||||
};
|
||||
by_id.insert(m.id.clone(), Arc::new(OpenAiProvider::new(&m, None)?));
|
||||
}
|
||||
|
||||
@@ -145,14 +145,6 @@ impl AppState {
|
||||
let live = Live::build(new_cfg.clone(), &self.secrets, self.router_status.clone())?;
|
||||
new_cfg.save(&self.config_path)?;
|
||||
self.live.store(Arc::new(live));
|
||||
|
||||
let cmf_rt = self.cmf.clone();
|
||||
let cmf_cfg = new_cfg.cmf.clone();
|
||||
tokio::spawn(async move {
|
||||
cmf_rt.stop().await;
|
||||
crate::cmf_runtime::manage(cmf_rt, cmf_cfg).await;
|
||||
});
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
Vendored
-167
@@ -397,170 +397,3 @@ tbody tr:hover td { background: var(--hover); }
|
||||
.mono.kv { display: flex; gap: 6px; padding: 1px 0; white-space: normal; overflow-wrap: anywhere; word-break: break-word; }
|
||||
.mono.kv .kv-k { color: var(--muted, #9aa4b2); flex: 0 0 auto; }
|
||||
.mono.kv .kv-v { font-weight: 600; overflow-wrap: anywhere; word-break: break-word; min-width: 0; }
|
||||
|
||||
/* ---- responsive shell + touch layout ----------------------------------- */
|
||||
.menu-toggle, .nav-close, .mobile-nav-backdrop { display: none; }
|
||||
.hamburger { width: 18px; display: flex; flex-direction: column; gap: 4px; }
|
||||
.hamburger i { display: block; height: 2px; border-radius: 2px; background: currentColor; }
|
||||
body.modal-open { overflow: hidden; }
|
||||
|
||||
@media (max-width: 980px) {
|
||||
.app { display: block; min-height: 100dvh; }
|
||||
.main { min-height: 100dvh; }
|
||||
.menu-toggle { display: inline-flex; flex: 0 0 auto; }
|
||||
.nav-close { display: inline-flex; margin-left: auto; flex: 0 0 auto; font-size: 24px; line-height: 1; }
|
||||
.brand-copy { min-width: 0; }
|
||||
|
||||
.sidebar {
|
||||
display: flex;
|
||||
position: fixed;
|
||||
inset: 0 auto 0 0;
|
||||
width: min(84vw, 310px);
|
||||
height: 100dvh;
|
||||
padding: max(18px, env(safe-area-inset-top)) 14px max(14px, env(safe-area-inset-bottom));
|
||||
z-index: 70;
|
||||
box-shadow: 18px 0 45px rgba(0, 0, 0, .42);
|
||||
transform: translateX(-104%);
|
||||
visibility: hidden;
|
||||
transition: transform .22s ease, visibility 0s linear .22s;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.app.nav-open .sidebar {
|
||||
transform: translateX(0);
|
||||
visibility: visible;
|
||||
transition: transform .22s ease;
|
||||
}
|
||||
.mobile-nav-backdrop {
|
||||
display: block;
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: 65;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
border: 0;
|
||||
background: rgba(3, 7, 18, .62);
|
||||
opacity: 0;
|
||||
visibility: hidden;
|
||||
pointer-events: none;
|
||||
transition: opacity .22s ease, visibility 0s linear .22s;
|
||||
}
|
||||
.app.nav-open .mobile-nav-backdrop {
|
||||
opacity: 1;
|
||||
visibility: visible;
|
||||
pointer-events: auto;
|
||||
transition: opacity .22s ease;
|
||||
}
|
||||
body.mobile-nav-open { overflow: hidden; }
|
||||
|
||||
.topbar { padding: 10px 18px; min-height: 58px; gap: 10px; }
|
||||
.topbar .title { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.topbar .right { gap: 8px; min-width: 0; }
|
||||
.content { max-width: none; padding: 22px 20px 52px; }
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
body { font-size: 14px; }
|
||||
.status-pills { display: none; }
|
||||
.topbar { padding: 9px 12px; }
|
||||
.content { padding: 18px 14px 42px; }
|
||||
|
||||
.page-head { margin-bottom: 16px; }
|
||||
.page-head > .flex { align-items: stretch; flex-direction: column; }
|
||||
.page-head > .flex > .btn { width: 100%; }
|
||||
.page-head p { line-height: 1.45; }
|
||||
|
||||
.card { padding: 15px; margin-bottom: 14px; border-radius: 12px; }
|
||||
.card .card-head { align-items: flex-start; flex-wrap: wrap; }
|
||||
.card .card-head .right { width: 100%; margin-left: 0; flex-wrap: wrap; }
|
||||
.grid { gap: 14px; }
|
||||
.kpis { gap: 10px; margin-bottom: 14px; }
|
||||
.kpi { padding: 12px; border-radius: 11px; }
|
||||
.kpi .big { font-size: 21px; }
|
||||
|
||||
.row { display: grid; grid-template-columns: minmax(0, 1fr); gap: 0; }
|
||||
.row > * { min-width: 0; width: 100%; }
|
||||
input, select, textarea { min-height: 44px; font-size: 16px; }
|
||||
input[type="checkbox"], input[type="radio"] { min-height: 0; }
|
||||
.check { min-height: 36px; margin-bottom: 8px; }
|
||||
.btn, .icon-btn { min-height: 44px; }
|
||||
.btn { padding: 10px 14px; }
|
||||
.btn.sm, .btn.small { min-height: 38px; padding: 8px 11px; }
|
||||
|
||||
.table-wrap {
|
||||
width: 100%;
|
||||
overflow-x: auto;
|
||||
overscroll-behavior-x: contain;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
scrollbar-width: thin;
|
||||
}
|
||||
.table-wrap table { min-width: max-content; }
|
||||
th, td { padding: 10px; }
|
||||
|
||||
.tierrow { align-items: flex-start; flex-direction: column; gap: 8px; }
|
||||
.tierrow .tname { width: auto; }
|
||||
.chips { width: 100%; }
|
||||
.togglerow { align-items: flex-start; }
|
||||
|
||||
.pg { gap: 14px; }
|
||||
.routecard .kv { flex-wrap: wrap; }
|
||||
.routecard .kv .v { max-width: 100%; overflow-wrap: anywhere; }
|
||||
.answer { padding: 12px; }
|
||||
.seg { max-width: 100%; overflow-x: auto; }
|
||||
.seg button { min-height: 40px; white-space: nowrap; }
|
||||
|
||||
.hf-grid { grid-template-columns: 1fr; max-height: 56dvh; }
|
||||
.job-head, .actions { align-items: stretch; flex-wrap: wrap; }
|
||||
.actions .btn { flex: 1 1 150px; }
|
||||
.key-row { flex-direction: column; }
|
||||
.key-row .btn { width: 100%; }
|
||||
.cmf-server-row { align-items: stretch !important; padding: 10px; border: 1px solid var(--border-soft); border-radius: 10px; }
|
||||
.cmf-server-row > input, .cmf-server-row > select { width: 100% !important; min-width: 0 !important; flex: 1 1 100% !important; }
|
||||
.cmf-server-row > .btn { flex: 1 1 calc(50% - 6px); }
|
||||
.cmf-server-row > .small { flex: 1 1 100%; }
|
||||
|
||||
.modal-backdrop { align-items: flex-end; padding: 8px 0 0; }
|
||||
.modal {
|
||||
max-width: none;
|
||||
max-height: calc(100dvh - 8px);
|
||||
overflow-y: auto;
|
||||
padding: 18px 15px max(14px, env(safe-area-inset-bottom));
|
||||
border-radius: 18px 18px 0 0;
|
||||
}
|
||||
.modal .modal-foot {
|
||||
position: sticky;
|
||||
bottom: calc(-1 * max(14px, env(safe-area-inset-bottom)));
|
||||
margin: 12px -15px 0;
|
||||
padding: 12px 15px max(14px, env(safe-area-inset-bottom));
|
||||
background: color-mix(in srgb, var(--card) 94%, transparent);
|
||||
backdrop-filter: blur(10px);
|
||||
border-top: 1px solid var(--border-soft);
|
||||
}
|
||||
.modal .modal-foot .btn { flex: 1; }
|
||||
|
||||
.toasts { left: 12px; right: 12px; bottom: max(12px, env(safe-area-inset-bottom)); }
|
||||
.toast { min-width: 0; width: 100%; }
|
||||
.login-wrap { align-items: flex-start; padding: max(20px, env(safe-area-inset-top)) 14px; }
|
||||
.login-card { padding: 22px 18px; margin-top: 8dvh; }
|
||||
}
|
||||
|
||||
@media (max-width: 480px) {
|
||||
.topbar .title { font-size: 14px; }
|
||||
.topbar .right { gap: 6px; }
|
||||
.lang-select { width: 54px; min-height: 44px; padding: 6px; }
|
||||
.icon-btn { width: 42px; height: 42px; }
|
||||
.kpis { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.kpi .lbl { line-height: 1.25; }
|
||||
.barlist .b { grid-template-columns: minmax(0, 1fr) auto; gap: 6px 10px; }
|
||||
.barlist .b .track { grid-column: 1 / -1; grid-row: 2; }
|
||||
.barlist .b .val { min-width: 0; }
|
||||
.empty { padding: 28px 12px; }
|
||||
}
|
||||
|
||||
@media (max-width: 360px) {
|
||||
.topbar .title { display: none; }
|
||||
.kpis { grid-template-columns: 1fr; }
|
||||
}
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.sidebar, .mobile-nav-backdrop { transition: none; }
|
||||
}
|
||||
|
||||
Vendored
+1
-1
@@ -19,6 +19,6 @@
|
||||
<noscript>This console requires JavaScript.</noscript>
|
||||
<div id="app"><div class="boot">Loading…</div></div>
|
||||
<div id="toasts" class="toasts"></div>
|
||||
<script type="module" src="/admin/js/app.js?v=3"></script>
|
||||
<script type="module" src="/admin/js/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Vendored
+2
-1
@@ -60,7 +60,8 @@ export const api = {
|
||||
listModels: () => j("GET", "/models"),
|
||||
createModel: (b) => j("POST", "/models", b),
|
||||
updateModel: (id, b) => j("PUT", "/models/" + encodeURIComponent(id), b),
|
||||
deleteModel: (id) => j("DELETE", "/models/" + encodeURIComponent(id)),
|
||||
deleteModel: (id, deleteFile = true) =>
|
||||
j("DELETE", "/models/" + encodeURIComponent(id) + "?delete_file=" + (deleteFile ? "true" : "false")),
|
||||
probeModel: (id) => j("POST", "/models/" + encodeURIComponent(id) + "/probe"),
|
||||
|
||||
// routing
|
||||
|
||||
Vendored
+5
-39
@@ -4,7 +4,7 @@ import { t, getLang, setLang, onLangChange, LANGS } from "./i18n.js";
|
||||
import { api, getToken, setToken, setUnauthorizedHandler } from "./api.js";
|
||||
import { renderLogin } from "./views/login.js";
|
||||
import { renderDashboard } from "./views/dashboard.js";
|
||||
import { renderModels } from "./views/models.js?v=3";
|
||||
import { renderModels } from "./views/models.js";
|
||||
import { renderImport } from "./views/import.js";
|
||||
import { renderRouting } from "./views/routing.js";
|
||||
import { renderProtocols } from "./views/protocols.js";
|
||||
@@ -28,17 +28,6 @@ export const appState = { meta: null, health: null, version: "" };
|
||||
// allaigate site — where router keys are issued and topped up
|
||||
export const SITE_URL = "https://api.allaigate.com/";
|
||||
|
||||
let mobileNavOpen = false;
|
||||
|
||||
function setMobileNav(open) {
|
||||
mobileNavOpen = !!open;
|
||||
const app = document.querySelector(".app");
|
||||
if (app) app.classList.toggle("nav-open", mobileNavOpen);
|
||||
document.body.classList.toggle("mobile-nav-open", mobileNavOpen);
|
||||
const toggle = document.querySelector(".menu-toggle");
|
||||
if (toggle) toggle.setAttribute("aria-expanded", String(mobileNavOpen));
|
||||
}
|
||||
|
||||
function currentRoute() {
|
||||
const id = location.hash.replace(/^#\/?/, "") || "dashboard";
|
||||
return ROUTES.find((r) => r.id === id) || ROUTES[0];
|
||||
@@ -53,18 +42,17 @@ const FAVICON =
|
||||
function sidebar(activeId) {
|
||||
return h(
|
||||
"aside",
|
||||
{ class: "sidebar", id: "sidebar", "aria-label": "Main navigation" },
|
||||
{ class: "sidebar", id: "sidebar" },
|
||||
h(
|
||||
"div",
|
||||
{ class: "brand" },
|
||||
h("img", { src: FAVICON, alt: "" }),
|
||||
h("div", { class: "brand-copy" }, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub"))),
|
||||
h("button", { class: "icon-btn nav-close", type: "button", title: "Close menu", "aria-label": "Close menu", onClick: () => setMobileNav(false) }, "×")
|
||||
h("div", {}, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub")))
|
||||
),
|
||||
...ROUTES.map((r) =>
|
||||
h(
|
||||
"a",
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id, onClick: () => setMobileNav(false) },
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id },
|
||||
h("span", { class: "ico" }, r.icon),
|
||||
t("nav." + r.id)
|
||||
)
|
||||
@@ -97,7 +85,7 @@ function statusPills() {
|
||||
: t("status.checking");
|
||||
return h(
|
||||
"div",
|
||||
{ class: "flex status-pills" },
|
||||
{ class: "flex" },
|
||||
h(
|
||||
"span",
|
||||
{ class: "pill" },
|
||||
@@ -148,19 +136,6 @@ function topbar(activeId) {
|
||||
return h(
|
||||
"header",
|
||||
{ class: "topbar" },
|
||||
h(
|
||||
"button",
|
||||
{
|
||||
class: "icon-btn menu-toggle",
|
||||
type: "button",
|
||||
title: "Menu",
|
||||
"aria-label": "Open menu",
|
||||
"aria-controls": "sidebar",
|
||||
"aria-expanded": String(mobileNavOpen),
|
||||
onClick: () => setMobileNav(!mobileNavOpen),
|
||||
},
|
||||
h("span", { class: "hamburger", "aria-hidden": "true" }, h("i"), h("i"), h("i"))
|
||||
),
|
||||
h("div", { class: "title" }, t("nav." + activeId)),
|
||||
h(
|
||||
"div",
|
||||
@@ -200,15 +175,12 @@ async function renderView() {
|
||||
|
||||
function renderShell() {
|
||||
const route = currentRoute();
|
||||
mobileNavOpen = false;
|
||||
document.body.classList.remove("mobile-nav-open");
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h(
|
||||
"div",
|
||||
{ class: "app" },
|
||||
sidebar(route.id),
|
||||
h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }),
|
||||
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" }))
|
||||
)
|
||||
);
|
||||
@@ -271,12 +243,6 @@ onLangChange(() => {
|
||||
window.addEventListener("hashchange", () => {
|
||||
if (document.querySelector(".app")) renderShell();
|
||||
});
|
||||
window.addEventListener("keydown", (e) => {
|
||||
if (e.key === "Escape" && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
window.addEventListener("resize", () => {
|
||||
if (window.innerWidth > 980 && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
setInterval(() => {
|
||||
if (document.querySelector(".app")) refreshHealth();
|
||||
}, 20000);
|
||||
|
||||
Vendored
+13
-23
@@ -132,6 +132,8 @@ export default {
|
||||
"models.form.secretMissing": "not set",
|
||||
"models.form.secretPlaceholder": "set or replace key (write-only)",
|
||||
"models.deleteConfirm": "Delete model \"{id}\"?",
|
||||
"models.deleteFile": "also delete the model file from disk",
|
||||
"models.deletedWithFile": "Deleted (file removed from disk)",
|
||||
"models.form.idOrModel": "Enter an id or a model name",
|
||||
"models.form.apiKey": "API Key",
|
||||
"models.form.apiKeyPlaceholder": "paste your API key",
|
||||
@@ -139,18 +141,6 @@ export default {
|
||||
"models.form.apiKeyHint": "Stored securely on the gateway; it is not shown again.",
|
||||
"models.form.keyStored": "A key is saved — leave blank to keep it, or paste a new one.",
|
||||
"models.form.keyFromEnv": "Using a key from an environment variable.",
|
||||
"models.form.paramsTitle": "Generation & Thinking Parameters (O(1), MTP, Temperature)",
|
||||
"models.form.temperature": "Temperature",
|
||||
"models.form.temperatureHint": "Generation temperature (0.0 = deterministic, 1.0+ = creative).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling threshold (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max Tokens",
|
||||
"models.form.maxTokensHint": "Maximum completion tokens to generate.",
|
||||
"models.form.thinkBudget": "Thinking Mode / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Reasoning token budget (0 = disabled, N > 0 = max reasoning tokens).",
|
||||
"models.form.systemPrompt": "Default System Prompt",
|
||||
"models.form.systemPromptHint": "Prepend automatically if request lacks a system prompt.",
|
||||
"models.form.systemPromptPlaceholder": "You are a helpful assistant...",
|
||||
"models.form.testKey": "Test & load models",
|
||||
"models.form.testing": "checking…",
|
||||
"models.form.foundModels": "{n} models loaded",
|
||||
@@ -274,11 +264,14 @@ export default {
|
||||
"settings.cmf.pick": "select a model",
|
||||
"settings.cmf.models": "Local models",
|
||||
"settings.cmf.modelsHelp": "Each row is one model the gateway runs (its own cortiq serve on its own port) and registers in the pool. Add as many as you have .cmf files and free ports.",
|
||||
"settings.cmf.rowHint": "id · .cmf file · port · threads · GPU. Changes apply on restart; deleting a row stops that server.",
|
||||
"settings.cmf.rowHint": "id · .cmf file · port · threads · GPU · O(1). Changes apply on restart; deleting a row stops that server.",
|
||||
"settings.cmf.o1Auto": "O(1): auto",
|
||||
"settings.cmf.o1Help": "O(1) attention at serve time: auto — follow the hint recorded in the .cmf; off — force exact attention (fixes incoherent output after a too-aggressive --o1 conversion, no re-convert needed); all/deepN — force-enable on those layers.",
|
||||
"settings.cmf.addServer": "+ Add local model",
|
||||
"settings.cmf.checkPort": "Check port",
|
||||
"settings.cmf.portFree": "port is free",
|
||||
"settings.cmf.portBusy": "port is in use",
|
||||
"settings.cmf.portOwn": "in use by this gateway's current server — freed on restart, replacing is fine",
|
||||
"settings.cmf.useSuggested": "use {port}",
|
||||
"settings.cmf.serving": "serving {model} on :{port}",
|
||||
"settings.cmf.intro": "Run models locally with the cortiq engine — works even offline or when the router is down.",
|
||||
@@ -386,7 +379,7 @@ export default {
|
||||
"import.failed": "Conversion failed — see the log.",
|
||||
"import.register": "Register as local model",
|
||||
"import.registered": "Registered as '{id}'. Run cortiq-server to serve it.",
|
||||
"import.jobs": "Downloads & Conversions",
|
||||
"import.jobs": "Conversions",
|
||||
"import.cancel": "Cancel",
|
||||
"import.cancelling": "Cancelling — cleaning up partial output…",
|
||||
"import.delete": "Delete",
|
||||
@@ -396,21 +389,12 @@ export default {
|
||||
"import.state.done": "done",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelled",
|
||||
"import.featuredTitle": "Featured Ready-to-Run .cmf Models",
|
||||
"import.readyCmfBadge": "Ready .cmf",
|
||||
"import.directDownload": "Download",
|
||||
"import.q.q8_2f": "8-bit two-field (𝒲×θ) — recommended: best quality/size.",
|
||||
"import.q.q8_row": "8-bit per-row — simple, robust.",
|
||||
"import.q.q4": "4-bit block — smallest, lower quality.",
|
||||
"import.q.q4_tiled": "4-bit tiled (Q4_TILED) — 1.66x faster on Metal GPU.",
|
||||
"import.q.vbit": "Variable 3–8 bit — per-expert bit budgets.",
|
||||
"import.q.q1t": "1.58-bit ternary (Q1T) — BitNet b1.58 with sparse outliers.",
|
||||
"import.q.q1s": "1-bit sparse (Q1S) — 1-bit base with holographic outliers.",
|
||||
"import.q.q1p": "1-bit PTQ (Q1P) — 1-bit with error diffusion.",
|
||||
"import.q.q1": "1-bit (Q1) — pure binary format.",
|
||||
"import.q.f16": "16-bit — no quantization.",
|
||||
"import.q.f32": "32-bit — full precision (large file).",
|
||||
|
||||
"import.adv.title": "Advanced settings",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Linear core",
|
||||
@@ -430,6 +414,12 @@ export default {
|
||||
"import.adv.o1DeepN": "N deep layers",
|
||||
"import.adv.o1Layers": "Layers (comma-separated)",
|
||||
"import.adv.o1mHint": "landmark budget (default 32)",
|
||||
"import.adv.o1mLabel": "Landmarks (--o1-m)",
|
||||
"import.adv.o1mHelp": "How many landmark tokens summarize the whole older history when attention is approximated. More = closer to exact attention but slower and more memory; fewer = faster but rougher. 32 is the validated sweet spot — change only if you know why.",
|
||||
"import.adv.o1windowLabel": "Exact window (--o1-window)",
|
||||
"import.adv.o1windowHelp": "The last N tokens always get exact (non-approximated) attention. This protects quality on the text the model just wrote/read. Bigger = better local quality, more memory. Default 128.",
|
||||
"import.adv.o1sinkLabel": "Sink keys (--o1-sink)",
|
||||
"import.adv.o1sinkHelp": "The first N tokens of the context are kept exact forever ('attention sinks'). Models lean on these anchors — keeping them exact stabilizes long generations. Default 4; rarely needs changing.",
|
||||
"import.adv.o1windowHint": "exact-window width (default 128)",
|
||||
"import.adv.o1sinkHint": "permanent exact sink keys (default 4)",
|
||||
};
|
||||
|
||||
Vendored
+13
-23
@@ -131,6 +131,8 @@ export default {
|
||||
"models.form.secretMissing": "не задан",
|
||||
"models.form.secretPlaceholder": "задать или заменить ключ (только запись)",
|
||||
"models.deleteConfirm": "Удалить модель \"{id}\"?",
|
||||
"models.deleteFile": "также удалить файл модели с диска",
|
||||
"models.deletedWithFile": "Удалено (файл стёрт с диска)",
|
||||
"models.form.idOrModel": "Укажите id или имя модели",
|
||||
"models.form.apiKey": "API-ключ",
|
||||
"models.form.apiKeyPlaceholder": "вставьте API-ключ",
|
||||
@@ -138,18 +140,6 @@ export default {
|
||||
"models.form.apiKeyHint": "Хранится безопасно на шлюзе; больше не показывается.",
|
||||
"models.form.keyStored": "Ключ сохранён — оставьте пустым, чтобы не менять, или вставьте новый.",
|
||||
"models.form.keyFromEnv": "Используется ключ из переменной окружения.",
|
||||
"models.form.paramsTitle": "Параметры генерации, O(1) и режим размышления",
|
||||
"models.form.temperature": "Температура",
|
||||
"models.form.temperatureHint": "Температура генерации (0.0 — точные факты, 1.0+ — творчество).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Порог сэмплирования токенов (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Макс. токенов (Max Tokens)",
|
||||
"models.form.maxTokensHint": "Максимальное количество генерируемых токенов.",
|
||||
"models.form.thinkBudget": "Режим размышления / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Бюджет токенов мыслей (0 = выключен, N > 0 = лимит рассуждений).",
|
||||
"models.form.systemPrompt": "Системный промпт по умолчанию",
|
||||
"models.form.systemPromptHint": "Автоматически подставляется, если запрос не содержит системное сообщение.",
|
||||
"models.form.systemPromptPlaceholder": "Вы — полезный ассистент...",
|
||||
"models.form.testKey": "Проверить и загрузить модели",
|
||||
"models.form.testing": "проверка…",
|
||||
"models.form.foundModels": "загружено моделей: {n}",
|
||||
@@ -273,11 +263,14 @@ export default {
|
||||
"settings.cmf.pick": "выберите модель",
|
||||
"settings.cmf.models": "Локальные модели",
|
||||
"settings.cmf.modelsHelp": "Каждая строка — отдельная модель: шлюз поднимает свой cortiq serve на своём порту и регистрирует её в пуле. Добавляйте столько, сколько есть .cmf-файлов и свободных портов.",
|
||||
"settings.cmf.rowHint": "id · файл .cmf · порт · потоки · GPU. Изменения применяются после перезапуска; удаление строки останавливает сервер.",
|
||||
"settings.cmf.rowHint": "id · файл .cmf · порт · потоки · GPU · O(1). Изменения применяются после перезапуска; удаление строки останавливает сервер.",
|
||||
"settings.cmf.o1Auto": "O(1): авто",
|
||||
"settings.cmf.o1Help": "O(1)-внимание при запуске: авто — использовать подсказку из .cmf; off — принудительно точное внимание (лечит бессвязный вывод после слишком агрессивной --o1 конвертации, без переконвертации); all/deepN — принудительно включить на этих слоях.",
|
||||
"settings.cmf.addServer": "+ Добавить модель",
|
||||
"settings.cmf.checkPort": "Проверить порт",
|
||||
"settings.cmf.portFree": "порт свободен",
|
||||
"settings.cmf.portBusy": "порт занят",
|
||||
"settings.cmf.portOwn": "занят текущим сервером шлюза — освободится после перезапуска, замена допустима",
|
||||
"settings.cmf.useSuggested": "занять {port}",
|
||||
"settings.cmf.serving": "работает: {model} на :{port}",
|
||||
"settings.cmf.intro": "Запускайте модели локально движком cortiq — работает даже офлайн или когда роутер недоступен.",
|
||||
@@ -385,7 +378,7 @@ export default {
|
||||
"import.failed": "Конвертация не удалась — смотрите лог.",
|
||||
"import.register": "Подключить как локальную модель",
|
||||
"import.registered": "Подключено как '{id}'. Запустите cortiq-server для обслуживания.",
|
||||
"import.jobs": "Загрузки / Конвертации",
|
||||
"import.jobs": "Конвертации",
|
||||
"import.cancel": "Отменить",
|
||||
"import.cancelling": "Отмена — удаляю недописанный файл…",
|
||||
"import.delete": "Удалить",
|
||||
@@ -395,21 +388,12 @@ export default {
|
||||
"import.state.done": "готово",
|
||||
"import.state.error": "ошибка",
|
||||
"import.state.cancelled": "отменено",
|
||||
"import.featuredTitle": "Рекомендуемые готовые .cmf модели",
|
||||
"import.readyCmfBadge": "Готовая .cmf",
|
||||
"import.directDownload": "Скачать",
|
||||
"import.q.q8_2f": "8-бит двухполевой (𝒲×θ) — рекомендуется: лучшее качество/размер.",
|
||||
"import.q.q8_row": "8-бит построчно — просто, надёжно.",
|
||||
"import.q.q4": "4-бит блочно — компактнее, ниже качество.",
|
||||
"import.q.q4_tiled": "4-бит тайловый (Q4_TILED) — х1.66 быстрее на Metal GPU.",
|
||||
"import.q.vbit": "Переменная 3–8 бит — per-expert бюджеты.",
|
||||
"import.q.q1t": "1.58-бит тернарный (Q1T) — BitNet b1.58 с оутлайерами.",
|
||||
"import.q.q1s": "1-бит спарс (Q1S) — 1-бит база с голографическими оутлайерами.",
|
||||
"import.q.q1p": "1-бит PTQ (Q1P) — 1-бит с переносом ошибки (error diffusion).",
|
||||
"import.q.q1": "1-бит (Q1) — чистый бинарный формат.",
|
||||
"import.q.f16": "16-бит — без квантования.",
|
||||
"import.q.f32": "32-бит — полная точность (большой файл).",
|
||||
|
||||
"import.adv.title": "Расширенные настройки",
|
||||
"import.adv.auto": "авто",
|
||||
"import.adv.linearCore": "Линейное ядро",
|
||||
@@ -429,6 +413,12 @@ export default {
|
||||
"import.adv.o1DeepN": "N глубоких слоёв",
|
||||
"import.adv.o1Layers": "Слои (через запятую)",
|
||||
"import.adv.o1mHint": "число опорных точек (по умолчанию 32)",
|
||||
"import.adv.o1mLabel": "Опорные точки (--o1-m)",
|
||||
"import.adv.o1mHelp": "Сколько опорных токенов представляют всю старую историю при аппроксимации внимания. Больше — ближе к точному вниманию, но медленнее и больше памяти; меньше — быстрее, но грубее. 32 — проверенное значение; меняйте, только если понимаете зачем.",
|
||||
"import.adv.o1windowLabel": "Точное окно (--o1-window)",
|
||||
"import.adv.o1windowHelp": "Последние N токенов всегда обрабатываются точным (не аппроксимированным) вниманием — это защищает качество на только что написанном/прочитанном тексте. Больше — лучше локальное качество, больше памяти. По умолчанию 128.",
|
||||
"import.adv.o1sinkLabel": "Sink-ключи (--o1-sink)",
|
||||
"import.adv.o1sinkHelp": "Первые N токенов контекста навсегда остаются точными («якоря внимания»). Модели опираются на эти якоря — их точность стабилизирует длинную генерацию. По умолчанию 4; менять почти никогда не нужно.",
|
||||
"import.adv.o1windowHint": "ширина точного окна (по умолчанию 128)",
|
||||
"import.adv.o1sinkHint": "постоянные точные sink-ключи (по умолчанию 4)",
|
||||
};
|
||||
|
||||
Vendored
+1
-14
@@ -23,10 +23,6 @@ export function clear(node) {
|
||||
while (node && node.firstChild) node.removeChild(node.firstChild);
|
||||
}
|
||||
|
||||
export function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
export function mount(node, ...children) {
|
||||
clear(node);
|
||||
for (const c of children.flat())
|
||||
@@ -50,14 +46,7 @@ export function toast(msg, kind = "info", ms = 3200) {
|
||||
// Modal dialog. `body` is a DOM node; `onSave` returns a Promise; returns nothing.
|
||||
export function modal(title, body, onSave, saveLabel) {
|
||||
const backdrop = h("div", { class: "modal-backdrop" });
|
||||
const onKeydown = (e) => {
|
||||
if (e.key === "Escape") close();
|
||||
};
|
||||
const close = () => {
|
||||
backdrop.remove();
|
||||
document.body.classList.remove("modal-open");
|
||||
document.removeEventListener("keydown", onKeydown);
|
||||
};
|
||||
const close = () => backdrop.remove();
|
||||
backdrop.addEventListener("click", (e) => {
|
||||
if (e.target === backdrop) close();
|
||||
});
|
||||
@@ -84,8 +73,6 @@ export function modal(title, body, onSave, saveLabel) {
|
||||
)
|
||||
);
|
||||
backdrop.appendChild(card);
|
||||
document.body.classList.add("modal-open");
|
||||
document.addEventListener("keydown", onKeydown);
|
||||
document.body.appendChild(backdrop);
|
||||
return { close };
|
||||
}
|
||||
|
||||
Vendored
+15
-60
@@ -2,7 +2,7 @@
|
||||
// live progress view. Search HF, pick quantization + advanced params, watch
|
||||
// the conversion (multiple in parallel, resumable across tab switches), then
|
||||
// register the result as a local model.
|
||||
import { h, mount, toast, opt } from "../ui.js";
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
@@ -10,27 +10,24 @@ const QUANTS = [
|
||||
["Q8_2F", "import.q.q8_2f"],
|
||||
["Q8_ROW", "import.q.q8_row"],
|
||||
["Q4_BLOCK", "import.q.q4"],
|
||||
["Q4_TILED", "import.q.q4_tiled"],
|
||||
["VBIT", "import.q.vbit"],
|
||||
["Q1T", "import.q.q1t"],
|
||||
["Q1S", "import.q.q1s"],
|
||||
["Q1P", "import.q.q1p"],
|
||||
["Q1", "import.q.q1"],
|
||||
["F16", "import.q.f16"],
|
||||
["F32", "import.q.f32"],
|
||||
];
|
||||
|
||||
const FEATURED_REPOS = [
|
||||
{ id: "infosave/Bonsai-1.7Bcmf", label: "Bonsai 1.7B CMF", badge: "1.7B · Q1", quant: "Q1" },
|
||||
{ id: "infosave/Bonsai-8B_2bit_cmf", label: "Bonsai 8B 2-bit CMF", badge: "8B · Q1T", quant: "Q1T" },
|
||||
{ id: "infosave/Bonsai-27Bcmf", label: "Bonsai 27B CMF", badge: "27B · Q1", quant: "Q1" },
|
||||
];
|
||||
|
||||
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control,
|
||||
// A small "?" badge with a hover/focus tooltip (same style as Settings).
|
||||
function help(text) {
|
||||
return text ? h("span", { class: "help", title: text, tabindex: "0", role: "img", "aria-label": text }, "?") : null;
|
||||
}
|
||||
function field(label, control, hint, helpText) {
|
||||
return h("label", { class: "field" },
|
||||
h("span", {}, label, helpText ? " " : null, help(helpText)),
|
||||
control,
|
||||
hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
function fmtNum(n) {
|
||||
if (n == null) return "—";
|
||||
if (n >= 1e6) return (n / 1e6).toFixed(1) + "M";
|
||||
@@ -150,9 +147,9 @@ export function renderImport() {
|
||||
field(t("import.adv.o1DeepN"), o1DeepIn),
|
||||
field(t("import.adv.o1Layers"), o1CustomIn));
|
||||
const o1Knobs = h("div", { class: "row", style: "display:none" },
|
||||
field("--o1-m", o1M, t("import.adv.o1mHint")),
|
||||
field("--o1-window", o1Window, t("import.adv.o1windowHint")),
|
||||
field("--o1-sink", o1Sink, t("import.adv.o1sinkHint")));
|
||||
field(t("import.adv.o1mLabel"), o1M, t("import.adv.o1mHint"), t("import.adv.o1mHelp")),
|
||||
field(t("import.adv.o1windowLabel"), o1Window, t("import.adv.o1windowHint"), t("import.adv.o1windowHelp")),
|
||||
field(t("import.adv.o1sinkLabel"), o1Sink, t("import.adv.o1sinkHint"), t("import.adv.o1sinkHelp")));
|
||||
o1Sel.addEventListener("change", () => {
|
||||
const on = !!o1Sel.value;
|
||||
o1Knobs.style.display = on ? "" : "none";
|
||||
@@ -367,46 +364,6 @@ export function renderImport() {
|
||||
);
|
||||
}
|
||||
|
||||
function featuredCard(item) {
|
||||
const card = h(
|
||||
"div",
|
||||
{ class: "hf-card featured-card" },
|
||||
h("div", { class: "row", style: "align-items:center; justify-content:space-between;" },
|
||||
h("div", {},
|
||||
h("div", { class: "hf-id" }, item.label),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, item.badge),
|
||||
h("span", { class: "muted" }, item.id)
|
||||
)
|
||||
),
|
||||
h("button", { class: "btn primary sm" }, "⬇ " + t("import.directDownload"))
|
||||
)
|
||||
);
|
||||
card.querySelector("button").addEventListener("click", async (e) => {
|
||||
e.stopPropagation();
|
||||
try {
|
||||
await api.startImport({
|
||||
repo: item.id,
|
||||
quant: item.quant || "READY_CMF",
|
||||
name: item.id.split("/").pop().toLowerCase(),
|
||||
});
|
||||
toast(t("import.started"), "ok");
|
||||
refreshJobs();
|
||||
jobsHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
|
||||
} catch (err) {
|
||||
toast(err.message, "err");
|
||||
}
|
||||
});
|
||||
return card;
|
||||
}
|
||||
|
||||
const featuredHost = h(
|
||||
"div",
|
||||
{ class: "card featured-section" },
|
||||
h("div", { class: "card-title" }, "⚡ " + t("import.featuredTitle")),
|
||||
h("div", { class: "hf-grid" }, ...FEATURED_REPOS.map(featuredCard))
|
||||
);
|
||||
|
||||
const root = h(
|
||||
"div",
|
||||
{},
|
||||
@@ -414,7 +371,6 @@ export function renderImport() {
|
||||
h("h2", {}, t("nav.import")),
|
||||
h("div", { class: "muted" }, t("import.subtitle"))),
|
||||
jobsHost, // progress panel on top, above the search
|
||||
featuredHost, // featured ready-to-run models
|
||||
h("div", { class: "card" }, searchIn, results),
|
||||
formHost
|
||||
);
|
||||
@@ -422,4 +378,3 @@ export function renderImport() {
|
||||
refreshJobs(); // resume: show any running/recent conversions immediately
|
||||
return root;
|
||||
}
|
||||
|
||||
|
||||
Vendored
+47
-112
@@ -1,5 +1,5 @@
|
||||
// Models — pool table with add/edit modal, probe, and per-model secret entry.
|
||||
import { h, mount, modal, toast, confirmDialog, money, opt } from "../ui.js";
|
||||
import { h, mount, modal, toast, money } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
@@ -7,6 +7,9 @@ import { appState } from "../app.js";
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
function keyBadge(src) {
|
||||
const map = { store: "store", env: "env", missing: "bad", none: "" };
|
||||
@@ -51,16 +54,7 @@ function modelForm(meta, existing) {
|
||||
const tierSel = h("select", {}, ...(meta.cost_tiers || ["cheap"]).map((c) => opt(c, c, c === m.cost_tier)));
|
||||
const priceInIn = h("input", { type: "number", step: "0.01", value: m.price_in ?? 0 });
|
||||
const priceOutIn = h("input", { type: "number", step: "0.01", value: m.price_out ?? 0 });
|
||||
const CAP_EMOJIS = {
|
||||
tools: "🔧 tools",
|
||||
vision: "👁️ vision",
|
||||
audio: "🎙️ audio",
|
||||
reasoning: "🧠 reasoning",
|
||||
json: "📄 json",
|
||||
caching: "⚡ caching"
|
||||
};
|
||||
|
||||
const capInputs = (meta.caps || Object.keys(CAP_EMOJIS)).map((c) => {
|
||||
const capInputs = (meta.caps || ["tools", "vision"]).map((c) => {
|
||||
const cb = h("input", { type: "checkbox", checked: (m.caps || []).includes(c) ? true : null });
|
||||
return { c, cb };
|
||||
});
|
||||
@@ -79,55 +73,25 @@ function modelForm(meta, existing) {
|
||||
capInputs.forEach(({ c, cb }) => { cb.checked = d.caps.includes(c); });
|
||||
keyIn.placeholder = d.needsKey ? t("models.form.apiKeyPlaceholder") : t("models.form.apiKeyNotNeeded");
|
||||
mount(keyStatus, "");
|
||||
// Automatically pre-fetch models (especially useful for OpenRouter which doesn't need a key)
|
||||
testKey(true);
|
||||
}
|
||||
providerSel.addEventListener("change", applyDefaults);
|
||||
if (!existing) applyDefaults();
|
||||
|
||||
let cachedModels = [];
|
||||
|
||||
const autofillData = () => {
|
||||
console.log("autofillData triggered. modelIn.value:", modelIn.value);
|
||||
console.log("cachedModels length:", cachedModels.length);
|
||||
if (!existing && (!idIn.value || idIn._auto)) {
|
||||
idIn.value = slugId(modelIn.value);
|
||||
idIn._auto = true;
|
||||
console.log("auto-filled idIn:", idIn.value);
|
||||
}
|
||||
const found = cachedModels.find((m) => (m.id || m) === modelIn.value);
|
||||
console.log("found model:", found);
|
||||
if (found) {
|
||||
if (found.price_in !== undefined) {
|
||||
priceInIn.value = parseFloat(Number(found.price_in).toFixed(4));
|
||||
priceOutIn.value = parseFloat(Number(found.price_out).toFixed(4));
|
||||
console.log("Updated prices:", priceInIn.value, priceOutIn.value);
|
||||
}
|
||||
if (found.caps && Array.isArray(found.caps)) {
|
||||
capInputs.forEach(({ c, cb }) => {
|
||||
cb.checked = found.caps.includes(c);
|
||||
});
|
||||
console.log("Updated caps:", found.caps);
|
||||
}
|
||||
}
|
||||
};
|
||||
modelIn.addEventListener("input", autofillData);
|
||||
modelIn.addEventListener("change", autofillData);
|
||||
|
||||
// Auto-fill the id from the model name (until the user edits id by hand).
|
||||
if (!existing) {
|
||||
modelIn.addEventListener("input", () => {
|
||||
if (!idIn.value || idIn._auto) {
|
||||
idIn.value = slugId(modelIn.value);
|
||||
idIn._auto = true;
|
||||
}
|
||||
});
|
||||
idIn.addEventListener("input", () => { idIn._auto = false; });
|
||||
} else {
|
||||
console.log("Editing existing model, fetching testKey(true)");
|
||||
testKey(true);
|
||||
}
|
||||
|
||||
async function testKey(silent = false) {
|
||||
if (!baseUrlIn.value.trim()) {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?"));
|
||||
return;
|
||||
}
|
||||
async function testKey() {
|
||||
if (!baseUrlIn.value.trim()) { mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?")); return; }
|
||||
testBtn.disabled = true;
|
||||
if (!silent) mount(keyStatus, t("models.form.testing"));
|
||||
mount(keyStatus, t("models.form.testing"));
|
||||
try {
|
||||
const r = await api.providerModels({
|
||||
provider: providerSel.value,
|
||||
@@ -136,61 +100,24 @@ function modelForm(meta, existing) {
|
||||
api_key_env: envFor(),
|
||||
});
|
||||
if (r.ok) {
|
||||
cachedModels = r.models || [];
|
||||
dl.replaceChildren(...cachedModels.map((mm) => h("option", { value: mm.id || mm })));
|
||||
|
||||
let msg = "✓ " + t("models.form.foundModels", { n: r.count });
|
||||
if (r.balance !== undefined) {
|
||||
msg += ` (Balance: $${r.balance.toFixed(4)})`;
|
||||
}
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, msg));
|
||||
if (!silent) modelIn.focus();
|
||||
dl.replaceChildren(...(r.models || []).map((mm) => h("option", { value: mm })));
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, "✓ " + t("models.form.foundModels", { n: r.count })));
|
||||
modelIn.focus();
|
||||
} else {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
}
|
||||
} catch (e) {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
}
|
||||
testBtn.disabled = false;
|
||||
}
|
||||
testBtn.addEventListener("click", () => testKey(false));
|
||||
keyIn.addEventListener("change", () => testKey(true));
|
||||
testBtn.addEventListener("click", testKey);
|
||||
|
||||
const keyHint =
|
||||
existing && existing.key_source === "store" ? t("models.form.keyStored")
|
||||
: existing && existing.key_source === "env" ? t("models.form.keyFromEnv")
|
||||
: t("models.form.apiKeyHint");
|
||||
|
||||
// ⚙ Generation & Thinking Parameters
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: m.temperature ?? "" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: m.top_p ?? "" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: m.max_tokens ?? "" });
|
||||
const thinkIn = h("input", { type: "number", min: "0", placeholder: "1024", value: m.think_budget ?? "" });
|
||||
const o1Sel = h("select", {},
|
||||
h("option", { value: "" }, t("import.adv.auto")),
|
||||
h("option", { value: "all" }, "all — O(1) Attention"),
|
||||
h("option", { value: "deep" }, "deep — O(1) Deep"),
|
||||
h("option", { value: "off" }, "off — Standard"));
|
||||
if (m.o1) o1Sel.value = m.o1;
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: m.skip_mtp ? true : null });
|
||||
const sysPromptIn = h("textarea", { rows: 2, placeholder: t("models.form.systemPromptPlaceholder"), value: m.system_prompt || "" });
|
||||
|
||||
const advParams = h(
|
||||
"details",
|
||||
{ class: "adv-params", style: "margin-top:12px;padding:10px;border:1px solid var(--border);border-radius:6px;" },
|
||||
h("summary", { style: "font-weight:600;cursor:pointer;" }, "⚙ " + t("models.form.paramsTitle")),
|
||||
h("div", { class: "row", style: "margin-top:8px" },
|
||||
field(t("models.form.temperature"), tempIn, t("models.form.temperatureHint")),
|
||||
field(t("models.form.topP"), topPIn, t("models.form.topPHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("models.form.maxTokens"), maxTokIn, t("models.form.maxTokensHint")),
|
||||
field(t("models.form.thinkBudget"), thinkIn, t("models.form.thinkBudgetHint"))),
|
||||
h("div", { class: "row" },
|
||||
field("O(1) Attention", o1Sel, t("import.adv.o1Hint")),
|
||||
field("MTP", h("label", { class: "check" }, skipMtpCb, t("import.adv.skipMtp")))),
|
||||
field(t("models.form.systemPrompt"), sysPromptIn, t("models.form.systemPromptHint"))
|
||||
);
|
||||
|
||||
const node = h(
|
||||
"div",
|
||||
{},
|
||||
@@ -201,8 +128,7 @@ function modelForm(meta, existing) {
|
||||
dl,
|
||||
h("div", { class: "row" }, field(t("models.form.kind"), kindSel), field(t("models.form.costTier"), tierSel)),
|
||||
h("div", { class: "row" }, field(t("models.form.priceIn"), priceInIn), field(t("models.form.priceOut"), priceOutIn)),
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap", style: "gap:10px" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check", style: "margin:0" }, cb, CAP_EMOJIS[c] || c)))),
|
||||
advParams
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check" }, cb, c))))
|
||||
);
|
||||
|
||||
const getValue = () => {
|
||||
@@ -221,13 +147,6 @@ function modelForm(meta, existing) {
|
||||
price_out: parseFloat(priceOutIn.value) || 0,
|
||||
api_key_env: hasKey ? keyEnv : null,
|
||||
caps: capInputs.filter(({ cb }) => cb.checked).map(({ c }) => c),
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: thinkIn.value !== "" ? parseInt(thinkIn.value) : null,
|
||||
o1: o1Sel.value || null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysPromptIn.value.trim() || null,
|
||||
},
|
||||
secret: typedKey,
|
||||
keyEnv,
|
||||
@@ -361,15 +280,31 @@ function row(m, meta, reload) {
|
||||
"button",
|
||||
{
|
||||
class: "btn sm danger",
|
||||
onClick: async () => {
|
||||
if (!(await confirmDialog(t("models.deleteConfirm", { id: m.id })))) return;
|
||||
try {
|
||||
await api.deleteModel(m.id);
|
||||
toast(t("toast.deleted"), "good");
|
||||
await reload();
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
onClick: () => {
|
||||
// local models may have a .cmf on disk — let the user decide its fate
|
||||
const isLocal = m.managed || /127\.0\.0\.1|localhost/.test(m.base_url || "");
|
||||
const fileCb = h("input", { type: "checkbox", checked: isLocal ? true : null });
|
||||
const body = h(
|
||||
"div",
|
||||
{},
|
||||
h("p", {}, t("models.deleteConfirm", { id: m.id })),
|
||||
isLocal
|
||||
? h("label", { class: "check" }, fileCb, t("models.deleteFile"))
|
||||
: null
|
||||
);
|
||||
modal(t("common.delete") + ": " + m.id, body, async () => {
|
||||
try {
|
||||
const r = await api.deleteModel(m.id, isLocal && fileCb.checked);
|
||||
toast(
|
||||
r.file_removed ? t("models.deletedWithFile") : t("toast.deleted"),
|
||||
"good"
|
||||
);
|
||||
await reload();
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
return false;
|
||||
}
|
||||
}, t("common.delete"));
|
||||
},
|
||||
},
|
||||
t("common.delete")
|
||||
|
||||
Vendored
+14
-98
@@ -234,6 +234,12 @@ export async function renderSettings() {
|
||||
const portIn = h("input", { type: "number", value: sv.port ?? 8090, style: "width:82px" });
|
||||
const threadsIn = h("input", { type: "number", min: 0, value: sv.threads ?? 8, style: "width:60px", title: t("settings.cmf.threadsHelp") });
|
||||
const gpuIn = h("input", { type: "checkbox", checked: sv.gpu ? true : null, title: t("settings.cmf.gpuHelp") });
|
||||
// serve-time O(1) override: off cures a too-aggressive --o1 conversion
|
||||
const o1Opts = ["", "off", "all", "deep4", "deep8", "deep12"];
|
||||
const o1In = h("select", { style: "width:110px", title: t("settings.cmf.o1Help") },
|
||||
...o1Opts.map((v) => h("option", { value: v, selected: (sv.o1 || "") === v ? true : null },
|
||||
v === "" ? t("settings.cmf.o1Auto") : v)));
|
||||
if (sv.o1 && !o1Opts.includes(sv.o1)) o1In.appendChild(h("option", { value: sv.o1, selected: true }, sv.o1));
|
||||
const checkBtn = h("button", { class: "btn small", type: "button" }, t("settings.cmf.checkPort"));
|
||||
const removeBtn = h("button", { class: "btn small danger", type: "button", title: t("common.delete") }, "✕");
|
||||
const st = h("span", { class: "small" });
|
||||
@@ -245,7 +251,11 @@ export async function renderSettings() {
|
||||
try {
|
||||
const r = await api.cmfPortCheck(p);
|
||||
if (r.available) mount(st, h("span", { style: "color:var(--good)" }, "✓ " + t("settings.cmf.portFree")));
|
||||
else {
|
||||
else if (r.managed) {
|
||||
// held by this gateway's own managed server — replacing the model on
|
||||
// this port is fine, it frees up on restart
|
||||
mount(st, h("span", { style: "color:var(--accent)" }, "ℹ " + t("settings.cmf.portOwn")));
|
||||
} else {
|
||||
const kids = [h("span", { style: "color:var(--bad)" }, "✗ " + t("settings.cmf.portBusy"))];
|
||||
if (r.suggested) { const b = h("button", { class: "btn small", type: "button" }, t("settings.cmf.useSuggested", { port: r.suggested })); b.addEventListener("click", () => { portIn.value = r.suggested; mount(st, ""); }); kids.push(b); }
|
||||
mount(st, h("span", {}, ...kids));
|
||||
@@ -253,103 +263,9 @@ export async function renderSettings() {
|
||||
} catch (e) { mount(st, String(e.message || e)); }
|
||||
checkBtn.disabled = false;
|
||||
});
|
||||
|
||||
// ── O(1) Nyström Attention ──
|
||||
// NOT a "reasoning mode". This is a memory-efficient streaming attention
|
||||
// kernel (Nyström/landmark) that replaces standard KV-cache attention,
|
||||
// giving O(1) memory per decode step instead of O(n).
|
||||
const o1Sel = h("select", { style: "width:120px", title: "O(1) Nyström attention layer spec: which layers replace KV-cache with streaming Nyström attention" },
|
||||
h("option", { value: "" }, "O(1) авто"),
|
||||
h("option", { value: "all" }, "O(1) все слои"),
|
||||
h("option", { value: "deep" }, "O(1) глубокие"),
|
||||
h("option", { value: "off" }, "O(1) выкл."));
|
||||
if (sv.o1) o1Sel.value = sv.o1;
|
||||
const o1mIn = h("input", { type: "number", min: "4", placeholder: "32", value: sv.o1_m ?? "", style: "width:55px", title: "Landmark budget (≥4, default 32). More landmarks ≠ better: m=64 measured worse." });
|
||||
const o1wIn = h("input", { type: "number", min: "1", placeholder: "128", value: sv.o1_window ?? "", style: "width:60px", title: "Exact-window width — main quality lever (default 128)" });
|
||||
const o1sinkIn = h("input", { type: "number", min: "0", placeholder: "4", value: sv.o1_sink ?? "", style: "width:50px", title: "Permanent exact sink keys, StreamingLLM discipline (default 4)" });
|
||||
|
||||
// ── Generation parameters ──
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: sv.temperature ?? "", style: "width:60px" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: sv.top_p ?? "", style: "width:60px" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: sv.max_tokens ?? "", style: "width:75px" });
|
||||
// think_budget = 0 means reasoning disabled (enable_thinking=false);
|
||||
// null = model decides; N>0 = token budget for the <think> block.
|
||||
const noThinkCb = h("input", { type: "checkbox", checked: (sv.think_budget === 0) ? true : null, title: "Отключить режим размышлений (think_budget=0, enable_thinking=false для Qwen3/3.5)" });
|
||||
const thinkIn = h("input", { type: "number", min: "1", placeholder: "авто", value: (sv.think_budget && sv.think_budget > 0) ? sv.think_budget : "", style: "width:75px", title: "Бюджет токенов на размышления (think_budget). Пустое = модель решает сама." });
|
||||
if (sv.think_budget === 0) thinkIn.disabled = true;
|
||||
noThinkCb.addEventListener("change", () => {
|
||||
thinkIn.disabled = noThinkCb.checked;
|
||||
if (noThinkCb.checked) thinkIn.value = "";
|
||||
});
|
||||
const sysIn = h("input", { placeholder: "System prompt (optional)", value: sv.system_prompt || "", style: "flex:1;min-width:200px" });
|
||||
|
||||
// ── MTP (Multi-Token Prediction): speculative decoding, NOT reasoning ──
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: sv.skip_mtp ? true : null, title: "Disable MTP speculative decoding (env CMF_MTP=0)" });
|
||||
|
||||
// ── Settings drawer ──
|
||||
const drawerStyle = "display:none;width:100%;margin-top:6px;padding:10px 12px;background:var(--bg-subtle);border-radius:6px;border:1px solid var(--border)";
|
||||
const groupStyle = "display:flex;flex-wrap:wrap;gap:8px;align-items:center;padding:4px 0";
|
||||
const labelStyle = "font-size:11px;color:var(--text-muted);font-weight:600;text-transform:uppercase;letter-spacing:0.5px;width:100%;margin-bottom:2px";
|
||||
|
||||
const drawer = h("div", { style: drawerStyle },
|
||||
// Group 1: O(1) Nyström Attention
|
||||
h("div", { style: labelStyle }, "O(1) Nyström Attention"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Слои:"), o1Sel,
|
||||
h("span", { class: "small muted" }, "m:"), o1mIn,
|
||||
h("span", { class: "small muted" }, "Window:"), o1wIn,
|
||||
h("span", { class: "small muted" }, "Sink:"), o1sinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 2: Generation
|
||||
h("div", { style: labelStyle }, "Генерация"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Temp:"), tempIn,
|
||||
h("span", { class: "small muted" }, "Top-P:"), topPIn,
|
||||
h("span", { class: "small muted" }, "Max токенов:"), maxTokIn
|
||||
),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "System prompt:"), sysIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 3: Reasoning (thinking)
|
||||
h("div", { style: labelStyle }, "Режим размышлений"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "Для моделей Qwen3/3.5: enable_thinking=false — модель отвечает напрямую без блока <think>" }, noThinkCb, "Отключить размышления"),
|
||||
h("span", { class: "small muted" }, "Think budget:"), thinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 4: Runtime
|
||||
h("div", { style: labelStyle }, "Runtime"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "MTP — Multi-Token Prediction (спекулятивное декодирование). Отключение замедляет генерацию, но стабилизирует output." }, skipMtpCb, "Отключить MTP")
|
||||
)
|
||||
);
|
||||
|
||||
const gearBtn = h("button", { class: "btn small", type: "button", title: "⚙ Настройки модели" }, "⚙");
|
||||
gearBtn.addEventListener("click", () => {
|
||||
drawer.style.display = drawer.style.display === "none" ? "block" : "none";
|
||||
});
|
||||
|
||||
const row = h("div", { class: "flex wrap cmf-server-row", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), gearBtn, removeBtn, st, drawer);
|
||||
const entry = { node: row, read: () => ({
|
||||
id: idIn.value.trim() || autoId(modelSel.value),
|
||||
model: modelSel.value.trim(),
|
||||
port: parseInt(portIn.value) || 8090,
|
||||
threads: parseInt(threadsIn.value) || 0,
|
||||
gpu: gpuIn.checked,
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: noThinkCb.checked ? 0 : (thinkIn.value !== "" ? parseInt(thinkIn.value) : null),
|
||||
o1: o1Sel.value || null,
|
||||
o1_m: o1mIn.value !== "" ? parseInt(o1mIn.value) : null,
|
||||
o1_window: o1wIn.value !== "" ? parseInt(o1wIn.value) : null,
|
||||
o1_sink: o1sinkIn.value !== "" ? parseInt(o1sinkIn.value) : null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysIn.value.trim() || null,
|
||||
}) };
|
||||
const row = h("div", { class: "flex wrap", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), o1In, removeBtn, st);
|
||||
const entry = { node: row, read: () => ({ id: idIn.value.trim() || autoId(modelSel.value), model: modelSel.value.trim(), port: parseInt(portIn.value) || 8090, threads: parseInt(threadsIn.value) || 0, gpu: gpuIn.checked, o1: o1In.value }) };
|
||||
removeBtn.addEventListener("click", () => { const i = serverRows.indexOf(entry); if (i >= 0) serverRows.splice(i, 1); row.remove(); });
|
||||
serverRows.push(entry);
|
||||
serversWrap.appendChild(row);
|
||||
|
||||
Reference in New Issue
Block a user