From 914ff42fc5d06be1a2295349c797d48526bb894c Mon Sep 17 00:00:00 2001 From: infosave2007 Date: Fri, 7 Aug 2026 18:34:00 +0300 Subject: [PATCH] =?UTF-8?q?release:=20v0.2.38=20=E2=80=94=20media=20genera?= =?UTF-8?q?tion,=20task-based=20routing,=20ops=20batch?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG.md | 41 +++++++++++++++++++++++++++++++++++++++++ Cargo.lock | 2 +- Cargo.toml | 2 +- README.md | 5 +++-- README.ru.md | 5 +++-- 5 files changed, 49 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2030af3..369ad7a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,47 @@ All notable changes to this project are documented here. The format is based on [Keep a Changelog](https://keepachangelog.com/). +## [0.2.38] - 2026-08-07 + +### Added +- **Image & video generation in the Playground**: three tabs now — Chat, Image, Video. Image runs + `cortiq imagine` over a local text-to-image `.cmf` (Lumina) with width/height/steps/CFG/seed, + live step progress and an honest ETA; the P6 PPM result is decoded to a canvas in the browser + with one-click PNG export. Video runs `cortiq animate` (MiniMax-H3) with frames/steps/quality + and optional first/last keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded + in JS and played on a canvas synced to the wav — no ffmpeg on either side (an mp4 remux happens + opportunistically when ffmpeg exists). Local `.cmf` files are classified chat/image/video via + `cortiq info`; media packs are never registered as chat servers. Requires cortiq ≥ 0.5.56. +- **Task-based routing**: `[routing.tasks]` maps a router task label to an ordered model list. + A matching rule's models lead the candidate chain and the complexity chain follows as fallback, + so a task pin never weakens failover. The Routing page builds the editor from the account's + live taxonomy (`GET /v1/taxonomies`), shows each task's share of 7-day traffic, and supports + custom labels. Responses whose chain was headed by a rule report `route_source "router+task-rule"`. +- **Generation gallery** under the media tabs (lazy PPM thumbnails, reopen with seed/steps, delete), + and generations **queue up** (with position shown) instead of being rejected. +- **Chat is a real dialogue**: history is resent, bubbles, Clear, Cmd/Ctrl+Enter — plus live + metrics: token counts, tok/s and time-to-first-token. +- **Exact stream token counts** when the model server provides usage; the gateway fallback counts + content chunks (exactly 1/token on cortiq serve) instead of chars/4, flagged as an estimate. +- **Jobs survive restarts**: import and media job lists persist to `/jobs-*.json` and are + reconciled on start — the Register button no longer vanishes because the gateway restarted. +- **Admin console, ops batch**: Logs viewer in the sidebar (500-line ring, filter/level/pause/copy); + a System panel on the dashboard (CPU sparkline + load, RAM incl. the gateway's RSS, free disk at + the models dir); "not enough disk / may not fit in RAM" badges on model cards before a download; + a Restart-gateway button; an update-available notice (crates.io checked twice a day); config + export as the real `gateway.toml` and TOML import (validated, hot-reloaded); "Connect your app" + snippets (curl/Python/JS) on the keys page; one-click `cortiq bench --json` on the Models page. +- **Docker ships the cortiq binary** (built in the builder stage, plus `libvulkan1`): local `.cmf` + chat models and media generation now work in containers, where auto-install never could. +- **Security**: an open admin console listening on 0.0.0.0 outside Docker warns in the log and + shows a dismissible banner. + +### Fixed +- The default config pointed at a placeholder `localhost:8080` router with a nonexistent taxonomy, + leaving every fresh install with a permanently red "Router: offline" badge. A fresh install now + targets the hosted allaigate router and degrades gracefully until a key is set. +- The wizard's Finish lands on the Playground instead of the Models list. + ## [0.2.37] - 2026-08-06 ### Added diff --git a/Cargo.lock b/Cargo.lock index d9c0379..c5adcbf 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570" [[package]] name = "cortiq-gateway" -version = "0.2.37" +version = "0.2.38" dependencies = [ "anyhow", "arc-swap", diff --git a/Cargo.toml b/Cargo.toml index c7fc0e1..f9d5aed 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "cortiq-gateway" -version = "0.2.37" +version = "0.2.38" edition = "2021" rust-version = "1.75" license = "Apache-2.0" diff --git a/README.md b/README.md index 9cc5930..66881a3 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,8 @@ One OpenAI-compatible endpoint → automatic model selection from your pool - **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)). - **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE). - **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators. -- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down. +- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down. Plus **per-task rules** (`[routing.tasks]`): pin translations to a local model and code to your strongest coder — the complexity chain stays as fallback ([docs](docs/ROUTING.md)). +- **Image & video generation.** The Playground drives local text-to-image (Lumina) and text-to-video (MiniMax-H3) `.cmf` models via `cortiq imagine`/`animate`: live progress with ETA, a generation gallery, PNG export, in-browser AVI playback — no ffmpeg anywhere. - **Semantic cache.** Optional embedding-based cache returns a stored answer for prompts semantically near a previous one — skipping the model call (hit-rate & savings on the dashboard). - **Built-in multilingual admin console.** Manage models, routing, protocols, keys and secrets from a web UI — **no hand-editing TOML, no restart** (changes are hot-reloaded). Available in **7 languages** (en, ru, de, fr, es, zh, tr) with light/dark themes. - **Live analytics.** Per-request stats (tokens, cost, latency, success rate, failovers), time-series charts, breakdowns by model/tier/task, and a Prometheus `GET /metrics` endpoint. @@ -137,7 +138,7 @@ The wizard takes you from nothing to a working local model: | **Language** | 7 languages; changeable later from the header | | **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress | | **Any HF model** | search HuggingFace and auto-convert to `.cmf` | -| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, and `routing.default` points at it | +| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, `routing.default` points at it, and you land in the Playground | No TOML editing, no separate "register" step, no manually started model server. Prefer to skip the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config — diff --git a/README.ru.md b/README.ru.md index fe7885d..3761ed5 100644 --- a/README.ru.md +++ b/README.ru.md @@ -39,7 +39,8 @@ - **Запуск без конфигурации.** `cortiq-gateway` без флагов: конфиг создаётся сам, занятый порт меняется на свободный, браузер открывается на мастере, который скачивает готовую локальную модель и заводит её в роутинг — без TOML, без токена, без отдельно запущенного сервера модели. - **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)). - **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE). -- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен. +- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен. Плюс **правила по задачам** (`[routing.tasks]`): переводы — в локальную модель, код — сильнейшему кодеру; цепочка по сложности остаётся резервом ([доки](docs/ROUTING.ru.md)). +- **Генерация картинок и видео.** Песочница запускает локальные text-to-image (Lumina) и text-to-video (MiniMax-H3) `.cmf` модели через `cortiq imagine`/`animate`: живой прогресс с ETA, галерея генераций, экспорт в PNG, проигрывание AVI прямо в браузере — без ffmpeg вообще. - **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема. - **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`. - **Playground.** Прогон промпта через живой пайплайн с разбором решения роутинга. @@ -129,7 +130,7 @@ cortiq-gateway | **Язык** | 7 языков; потом меняется в шапке | | **Готовые модели** | `.cmf` модели [`infosave`](https://huggingface.co/infosave), список читается с HuggingFace вживую — выбираете квантизацию и видите реальный прогресс загрузки | | **Любая модель с HF** | поиск по HuggingFace и автоконвертация в `.cmf` | -| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё | +| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё — и вы попадаете в Песочницу | Никаких правок TOML, отдельного шага «зарегистрировать» и вручную запущенного сервера модели. Мастер не нужен? Добавляйте модели из панели (**Модели**, **Импорт модели**) или пишите конфиг