chore: release 0.2.39
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
This commit is contained in:
@@ -3,6 +3,48 @@
|
||||
All notable changes to this project are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [0.2.39] - 2026-08-07
|
||||
|
||||
### Added
|
||||
- **Benchmark, redesigned into the console** (⏱ sidebar entry): two explicit modes. **Smart
|
||||
routing** (default) sends every probe through `cortiq-auto` and judges the ROUTER — task label
|
||||
vs expected (✓/✗), complexity tier, chosen model with per-request cost, task-rule badge —
|
||||
with KPIs for answer quality, classification accuracy, TTFT, tok/s and money spent, plus a
|
||||
"requests went to" per-model distribution. **Individual models** compares pinned models
|
||||
head-to-head. Results render live row-by-row, runs land in a browser-side history (per-run ×
|
||||
and Clear-all), custom tests supported, ⚡ quick subset (7) vs full suite (17).
|
||||
- **Hard probes that exercise the high tier**: two route-judged tests (science / ops pools)
|
||||
generated fresh each run — interdisciplinary prompts rate high reliably while fixed wordings
|
||||
decay with the router's novelty signal. Pass = the router said `high` and the request escalated;
|
||||
the tiny token budget keeps them at cents even on the priciest model.
|
||||
- **One-paste harness setup** on the API-keys page: Claude Code (native Anthropic `/v1/messages`,
|
||||
now on by default, one-click enable for older configs), Codex, OpenCode, Continue and Cline —
|
||||
each pre-filled with the gateway URL and the freshly created key. The gateway detects harnesses
|
||||
already installed on its machine (✓ on the tab, "already configured for this gateway" status).
|
||||
- **Stale-tab detection**: open consoles notice a gateway restart via the meta start marker and
|
||||
show a reload banner instead of silently running an old interface.
|
||||
- Log viewer Clear button (`DELETE /admin/api/logs`).
|
||||
|
||||
### Changed
|
||||
- **High complexity escalates over task rules**: at tier `high` the tier chain leads and the task
|
||||
rule demotes to fallback — a `code → cheap-model` rule no longer pins the hardest prompts to
|
||||
the cheap model. Low/medium keep task-rule-first semantics (documented in docs/ROUTING).
|
||||
- **Honest stream costs**: the final cost rides a closing SSE event (`cortiq_cost_usd`) since the
|
||||
initial header is written before generation; money renders with 4 decimals (last two smaller)
|
||||
and grows precision below $0.0001 instead of collapsing to $0.0000.
|
||||
- **Oracle-sized timeouts**: router default 800 ms → 15 s (hosted escalations take ~10 s);
|
||||
provider clients gained a 10 s connect timeout (dead upstreams fail over fast) with no overall
|
||||
cap so long generations and streams are never cut.
|
||||
- Benchmark fairness: +256-token reasoning headroom (thinking models aren't cut mid-`<think>`),
|
||||
router degradation shown as "⚠ router degraded" and excluded from accuracy, "budget spent on
|
||||
reasoning" instead of a misleading "empty answer", creative-writing budget 96 → 512.
|
||||
- New-key dialog simplified to account / rate limit / optional custom value.
|
||||
|
||||
### Fixed
|
||||
- `cortiq bench` results showed "error" with newer cortiq releases that pretty-print `--json`:
|
||||
the parser now finds the last balanced JSON object anywhere in the output, and tok/s prefers
|
||||
steady decode speed (the old heuristic could report `allocs_per_token` as tok/s).
|
||||
|
||||
## [0.2.38] - 2026-08-07
|
||||
|
||||
### Added
|
||||
|
||||
Reference in New Issue
Block a user