infosave2007
5759de153b
- cache.rs: cosine-similarity store (threshold, TTL, ring buffer), partitioned by routing signature; tracks hit rate and model cost saved - pipeline: embed routing text, look up before the model call (route_source=cache on hit), insert successful responses; stats record cache hits - config [cache] + admin settings card; dashboard shows hit-rate and savings - embedding model resolved from the pool (or first kind="embedding")
3.0 KiB
3.0 KiB
Changelog
All notable changes to this project are documented here. The format is based on Keep a Changelog.
[Unreleased]
Added
- Streaming (SSE) end-to-end:
POST /v1/chat/completionswith"stream": trueforwards provider SSE chunks verbatim, withX-Cortiq-*routing headers and token usage tapped from the final chunk for statistics. Failover applies before the first byte. Live streaming output in the admin Playground (/admin/api/test/stream). - Cross-platform Windows binaries in releases (
x86_64-pc-windows-msvc) and a cross-platform CI test matrix (ubuntu/windows/macos); secure token RNG viagetrandom. - Anthropic provider (outbound):
POST /v1/messageswithsystemextraction,x-api-key+anthropic-versionheaders, and streaming that translates Anthropic SSE events into the unified OpenAI wire format. - Anthropic inbound adapter:
POST /v1/messages(non-streaming JSON and streaming SSE), so Claude-native clients can target the gateway. - Embeddings:
POST /v1/embeddingsinbound + providerembed(); embedding models are now resolvable from the pool. - Semantic cache: embedding-based cache that returns a stored answer for prompts
semantically near a previous one (cosine threshold, TTL, ring buffer), skipping the
model call. Partitioned by routing signature; hit-rate and cost saved are tracked and
shown on the dashboard; configurable from Settings (
[cache]).
[0.1.0] - 2026-06-30
Added
- Embedded admin web console at
/admin— a build-free single-page app compiled into the binary viarust-embed. Manage models, routing, protocols, virtual keys and secrets from the browser; changes are hot-reloaded (no restart). - Multilingual UI in 7 languages (en, ru, de, fr, es, zh, tr) with browser auto-detection, plus light/dark themes.
- Admin API under
/admin/api/*(Bearer admin token): config, models (CRUD + probe), routing, protocols, settings, keys, secrets, stats, requests, and a playgroundtestendpoint. - Hot config reload — config-derived state lives behind
arc-swap; writes validate → atomically rewrite the TOML → swap, with no restart. - Request statistics — in-memory aggregates, per-minute time buckets, and a ring buffer of recent requests, appended to a JSONL file and replayed on startup.
- Prometheus
GET /metricsendpoint. - Secret store (
config/secrets.toml, mode0600) — provider keys can be set from the UI without writing them intogateway.toml; values are never returned by the API. - English documentation as the primary language, with Russian translations
(
README.ru.md,docs/*.ru.md).
Changed
- The Anthropic provider now returns a clear error instead of panicking when called (the adapter is still planned for v0.2).
[0.0.1]
- Initial skeleton: OpenAI Chat Completions inbound adapter, OpenAI-compatible outbound provider, routing table with fallback and graceful degradation, and cost/token accounting.