feat(import): O(1) attention options in Advanced (cortiq >= 0.2.0) · 243d8f4c7d - cortiq-gateway

feat(import): O(1) attention options in Advanced (cortiq >= 0.2.0)

Docker / publish (push) Canceled after 0s

Details

Release / crates-io (push) Canceled after 0s

Details

Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s

Details

Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s

Details

Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s

Details

- New Advanced controls on the import page: O(1) attention spec (off / all /
  deep N / explicit layer list) plus the --o1-m / --o1-window / --o1-sink knobs
  with validated defaults (32 / 128 / 4) and explanatory hints.
- Backend passes --o1* to the native `cortiq convert`; fails fast with clear
  messages when combined with Python-only options, used on GGUF imports, or
  when the installed cortiq predates 0.2.0 (the hint needs the new runtime).
- The hint is recorded in the .cmf header (weights byte-identical); the runtime
  applies it at load. Verified end-to-end: converted a model with --o1 all and
  the runtime logged "o1 nystrom attention: 2/2 layer(s)".

This commit is contained in:

infosave2007

2026-07-14 10:16:54 +03:00

parent 4f72711a1e

commit 243d8f4c7d

7 changed files with 126 additions and 3 deletions

Cargo.lock

Generated

+1 -1

View File

@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
 [[package]]
 name = "cortiq-gateway"
 version = "0.2.14"
 version = "0.2.15"
 dependencies = [
  "anyhow",
  "arc-swap",