github/cortiq-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: universal LLM gateway with embedded multilingual admin console

Browse Source 
- OpenAI-compatible gateway with intelligent routing via cortiq-router (allaigate)
- Embedded build-free admin SPA at /admin: 7 languages (en/ru/de/fr/es/zh/tr),
  light/dark themes, hot config reload (no restart)
- Admin API, per-request statistics + Prometheus /metrics, secret store, playground
- English documentation (primary) with Russian translations (*.ru.md)
This commit is contained in:
infosave2007
2026-06-30 10:51:37 +03:00
commit cbc385f6c7
63 changed files with 11339 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitattributes
+7
View File
@@ -0,0 +1,7 @@
# Normalize line endings
* text=auto eol=lf
# The embedded admin SPA is first-party but ships as a UI asset; exclude it from
# GitHub's language statistics so the repo is correctly identified as Rust.
web/** linguist-vendored
docs/screenshots/** linguist-vendored
.github/workflows/ci.yml
+24
View File
@@ -0,0 +1,24 @@
name: CI
on:
push:
branches: [ main, master ]
pull_request:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy
- uses: Swatinem/rust-cache@v2
- name: Format
run: cargo fmt --all -- --check
- name: Clippy
run: cargo clippy --all-targets -- -D warnings
- name: Build
run: cargo build --release
- name: Test
run: cargo test
.gitignore
+10
View File
@@ -0,0 +1,10 @@
/target
**/*.rs.bk
# Cargo.lock is committed: this is an application (binary), so we want reproducible builds.
config/gateway.toml
config/secrets.toml
config/stats.jsonl
config/*.toml.tmp
.env
*.log
.DS_Store
CHANGELOG.md
+35
View File
@@ -0,0 +1,35 @@
# Changelog
All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [Unreleased]
### Added
- **Embedded admin web console** at `/admin` — a build-free single-page app compiled
into the binary via `rust-embed`. Manage models, routing, protocols, virtual keys
and secrets from the browser; changes are hot-reloaded (no restart).
- **Multilingual UI** in 7 languages (en, ru, de, fr, es, zh, tr) with browser
auto-detection, plus light/dark themes.
- **Admin API** under `/admin/api/*` (Bearer admin token): config, models (CRUD +
probe), routing, protocols, settings, keys, secrets, stats, requests, and a
playground `test` endpoint.
- **Hot config reload** — config-derived state lives behind `arc-swap`; writes
validate → atomically rewrite the TOML → swap, with no restart.
- **Request statistics** — in-memory aggregates, per-minute time buckets, and a ring
buffer of recent requests, appended to a JSONL file and replayed on startup.
- **Prometheus `GET /metrics`** endpoint.
- **Secret store** (`config/secrets.toml`, mode `0600`) — provider keys can be set
from the UI without writing them into `gateway.toml`; values are never returned by
the API.
- English documentation as the primary language, with Russian translations
(`README.ru.md`, `docs/*.ru.md`).
### Changed
- The Anthropic provider now returns a clear error instead of panicking when called
(the adapter is still planned for v0.2).
## [0.0.1]
- Initial skeleton: OpenAI Chat Completions inbound adapter, OpenAI-compatible
outbound provider, routing table with fallback and graceful degradation, and
cost/token accounting.
CONTRIBUTING.md
+46
View File
@@ -0,0 +1,46 @@
*Read this in [Русский](CONTRIBUTING.ru.md).*
# Contributing to Cortiq Gateway
Thank you for your interest in the project! This is an open gateway and we welcome contributions.
## Getting started
1. Read [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — design overview and roadmap.
2. Review [docs/PROTOCOLS.md](docs/PROTOCOLS.md) and [docs/ROUTING.md](docs/ROUTING.md).
3. The current phase is **Phase 0** (skeleton). Good entry-point tasks are labeled
`good first issue`.
## How to add...
- **A new inbound protocol** — implement the `InboundProtocol` trait in `src/protocols/`,
translate to/from the canonical model (`ChatRequest`/`ChatResponse`), and add the
corresponding flag in `[protocols]`. Do not touch the providers.
- **A new provider** — implement the `Provider` trait in `src/providers/`. Most local
servers are OpenAI-compatible, so reusing `openai.rs` is often sufficient.
- **A new routing policy** — extend `src/routing.rs` and the `[routing.policy]` section.
Core principle: N protocols × M providers communicate **only** through the canonical model —
no direct protocol↔provider translations.
## Style and checks
```bash
cargo fmt --all
cargo clippy --all-targets -- -D warnings
cargo test
```
- Match the style of the surrounding code.
- Secrets go only through `*_env` variables — never commit them to the repository.
- New functionality must be accompanied by tests and updates to `docs/`.
## Pull requests
- Keep PRs small and focused. One PR = one logical unit of change.
- In the description: what, why, and how it was tested.
- Link the related issue if one exists.
## License
Contributions are accepted under [Apache-2.0](LICENSE).
CONTRIBUTING.ru.md
+45
View File
@@ -0,0 +1,45 @@
*Read this in [English](CONTRIBUTING.md).*
# Contributing to Cortiq Gateway
Спасибо за интерес к проекту! Это открытый шлюз, и мы рады контрибьюциям.
## С чего начать
1. Прочитайте [docs/ARCHITECTURE.md](docs/ARCHITECTURE.ru.md) — устройство и дорожная карта.
2. Посмотрите [docs/PROTOCOLS.md](docs/PROTOCOLS.ru.md) и [docs/ROUTING.md](docs/ROUTING.ru.md).
3. Текущая фаза — **Phase 0** (скелет). Хорошие первые задачи помечены `good first issue`.
## Как добавить...
- **Новый входящий протокол** — реализуйте трейт `InboundProtocol` в `src/protocols/`,
переведите его в/из канонической модели (`ChatRequest`/`ChatResponse`), добавьте флаг
в `[protocols]`. Не трогайте провайдеры.
- **Нового провайдера** — реализуйте трейт `Provider` в `src/providers/`. Большинство
локальных серверов OpenAI-совместимы → часто достаточно переиспользовать `openai.rs`.
- **Новую политику роутинга** — расширьте `src/routing.rs` и секцию `[routing.policy]`.
Принцип: N протоколов × M провайдеров общаются **только** через каноническую модель —
никаких прямых переводов «протокол↔провайдер».
## Стиль и проверки
```bash
cargo fmt --all
cargo clippy --all-targets -- -D warnings
cargo test
```
- Пишите код в стиле окружающего кода.
- Секреты — только через `*_env`, никогда в репозиторий.
- Новый функционал сопровождайте тестами и обновлением docs/.
## Pull requests
- Небольшие, сфокусированные PR. Один PR — одна логическая единица.
- В описании: что, зачем, как протестировано.
- Линкуйте issue, если есть.
## Лицензия
Контрибьюции принимаются под [Apache-2.0](LICENSE).
Cargo.lock
Generated
+2162
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+49
View File
@@ -0,0 +1,49 @@
[package]
name = "cortiq-gateway"
version = "0.0.1"
edition = "2021"
license = "Apache-2.0"
description = "Universal LLM gateway with intelligent routing via cortiq-router"
repository = "https://github.com/infosave2007/cortiq-gateway"
readme = "README.md"
keywords = ["llm", "gateway", "proxy", "openai", "routing"]
categories = ["web-programming::http-server"]
[dependencies]
# async runtime + HTTP server
tokio = { version = "1", features = ["full"] }
axum = { version = "0.7", features = ["macros"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["limit", "trace", "cors"] }
# HTTP client to providers and to cortiq-router (rustls, no OpenSSL)
reqwest = { version = "0.12", default-features = false, features = ["json", "stream", "rustls-tls"] }
# serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
toml = "0.8"
# hot-reloadable shared state + embedded admin SPA
arc-swap = "1"
rust-embed = "8"
mime_guess = "2"
# streaming / async utils
futures = "0.3"
async-trait = "0.1"
bytes = "1"
tokio-stream = "0.1"
# CLI / config
clap = { version = "4", features = ["derive"] }
# errors / logging / metrics
thiserror = "1"
anyhow = "1"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[profile.release]
lto = true
codegen-units = 1
LICENSE
+179
View File
@@ -0,0 +1,179 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or Derivative
Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Copyright 2026 Cortiq
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
README.md
+222
View File
@@ -0,0 +1,222 @@
# Cortiq Gateway
**English** · [Русский](README.ru.md)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
![Rust](https://img.shields.io/badge/Rust-2021-orange.svg)
![Status](https://img.shields.io/badge/status-active-success.svg)
**A universal LLM gateway with intelligent routing.**
One OpenAI-compatible endpoint → automatic model selection from your pool
(cheap local + expensive hosted) based on the task type and complexity decided by
[allaigate / cortiq-router](https://api.allaigate.com).
> In your agent/SDK you only change `base_url` — and you get "smart" routing across
> models. No model-selection logic on the client side.
```
┌─────────────┐ OpenAI / Anthropic / MCP ┌──────────────────┐
│ Agent / │ ─────────────────────────────▶│ Cortiq Gateway │
│ developer │ (standard protocol) │ (this project) │
└─────────────┘ ◀─────────────────────────────└──────────────────┘
response + metadata │ │
│ /v1/route│ call LLM
▼ ▼
┌──────────────┐ ┌─────────────────────┐
│ cortiq-router│ │ Model pool │
│ (task type, │ │ • local llama.cpp │
│ complexity) │ │ • ollama / vLLM │
└──────────────┘ │ • OpenAI / Claude │
└─────────────────────┘
```
---
## ✨ Highlights
- **Drop-in OpenAI API.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`.
- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down.
- **Built-in multilingual admin console.** Manage models, routing, protocols, keys and secrets from a web UI — **no hand-editing TOML, no restart** (changes are hot-reloaded). Available in **7 languages** (en, ru, de, fr, es, zh, tr) with light/dark themes.
- **Live analytics.** Per-request stats (tokens, cost, latency, success rate, failovers), time-series charts, breakdowns by model/tier/task, and a Prometheus `GET /metrics` endpoint.
- **Playground.** Send a prompt through the live pipeline and inspect the routing decision.
- **Single self-contained binary.** The SPA is embedded into the Rust binary — nothing extra to deploy.
- **Secrets stay in the gateway.** Agents hold only a virtual gateway key; provider keys live in the gateway and never leak to clients.
---
## 🖥️ Admin console
![Dashboard](docs/screenshots/dashboard.png)
The gateway ships with an embedded web console at **`/admin`**:
| | |
|---|---|
| ![Models](docs/screenshots/models.png) | ![Routing](docs/screenshots/routing.png) |
| **Models** — add/edit/probe models, manage provider keys | **Routing** — visual tier editor (ordered, drag-to-reorder) |
| ![Playground](docs/screenshots/playground.png) | ![Dashboard](docs/screenshots/dashboard.png) |
| **Playground** — test the live pipeline, see the decision | **Dashboard** — traffic, cost, latency analytics |
```bash
cargo run --release -- --config config/gateway.toml --admin-token <YOUR_TOKEN>
# open http://localhost:9000/admin?token=<YOUR_TOKEN>
```
If `--admin-token` / `[admin].token_env` are not set, a token is generated at startup
and printed to the log. All `/admin/api/*` endpoints require a Bearer admin token;
secret values are never returned by the API (only their presence: `store` / `env` / `missing`).
---
## 🚀 Quick start
```bash
# 1. start a router (the hosted allaigate router, or a local cortiq-router)
# it listens e.g. on http://localhost:8080 (or https://api.allaigate.com)
# 2. describe your model pool
cp config/gateway.example.toml config/gateway.toml
$EDITOR config/gateway.toml
# 3. run the gateway
cargo run --release -- --config config/gateway.toml
# Gateway listens on 0.0.0.0:9000 and serves the admin console at /admin
```
Now any OpenAI client works through the gateway:
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:9000/v1", api_key="sk-gw-...")
resp = client.chat.completions.create(
model="cortiq-auto", # ← magic model = "choose for me"
messages=[{"role": "user", "content": "Solve x^2 - 5x + 6 = 0"}],
)
print(resp.choices[0].message.content)
# response headers show what the gateway picked:
# X-Cortiq-Task-Label: math
# X-Cortiq-Complexity-Tier: low
# X-Cortiq-Selected-Model: local-qwen
```
`model: "cortiq-auto"` enables routing. Any **real** model name from the config
(`"gpt-4o-mini"`, `"local-qwen"`) is a direct passthrough, without routing.
> **Using the hosted allaigate router?** Set `url = "https://138.226.222.209"`,
> `verify_tls = false`, `taxonomy_id = "data-assistant"`, and a `cortiq_…` key in
> `CORTIQ_ROUTER_KEY`. On hard prompts the router escalates to an oracle (~10 s), so
> set `timeout_ms = 12000+` — otherwise the gateway gracefully degrades to the default model.
---
## Request flow
1. Client → `POST /v1/chat/completions` (or Anthropic/MCP) with `model: "cortiq-auto"`.
2. The gateway extracts the **text to route on** (strategy is configurable, see [docs/ROUTING.md](docs/ROUTING.md)).
3. Gateway → `cortiq-router /v1/route` → gets `task_label` + `complexity.tier`.
4. Gateway selects a model from the pool via the routing table (with a fallback order).
5. Gateway → the selected model's provider (translating the protocol if needed), returns the response.
6. Gateway attaches routing metadata in headers/`usage` and records cost & statistics.
Details — [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
---
## Supported protocols (inbound)
Each adapter is toggled in the config — take only what you need.
| Protocol | Endpoint | Status |
|---|---|---|
| OpenAI Chat Completions | `POST /v1/chat/completions` | ✅ implemented |
| OpenAI Completions (legacy) | `POST /v1/completions` | planned |
| OpenAI Embeddings | `POST /v1/embeddings` | planned |
| OpenAI Models | `GET /v1/models` | planned |
| Anthropic Messages | `POST /v1/messages` | planned |
| MCP (Model Context Protocol) | JSON-RPC / SSE | planned |
| Native passthrough | `POST /route` | planned |
## Supported providers (outbound)
| Provider | Covers |
|---|---|
| `openai` (OpenAI-compatible) | OpenAI, OpenRouter, Together, Groq, **vLLM, llama.cpp server, LM Studio, Ollama (`/v1`)** |
| `anthropic` | Claude Messages API (planned) |
| `ollama` | native Ollama API |
| `http` | arbitrary HTTP endpoint (custom adapter) |
Most local servers already expose an OpenAI-compatible API — so the `openai`
adapter makes the gateway **universal** with almost no effort.
---
## Configuration
```toml
listen = "0.0.0.0:9000"
[router]
url = "http://localhost:8080"
api_key_env = "CORTIQ_ROUTER_KEY"
[[models]]
id = "local-qwen"
provider = "openai"
base_url = "http://localhost:8000/v1"
model = "qwen2.5-7b-instruct"
cost_tier = "cheap"
[[models]]
id = "claude-opus"
provider = "anthropic"
base_url = "https://api.anthropic.com"
model = "claude-opus-4"
api_key_env = "ANTHROPIC_API_KEY"
cost_tier = "expensive"
[routing]
low = ["local-qwen"]
high = ["claude-opus", "local-qwen"]
default = "local-qwen"
```
Full example — [config/gateway.example.toml](config/gateway.example.toml).
Routing & cost-aware selection — [docs/ROUTING.md](docs/ROUTING.md).
Everything in the config is also editable from the admin console at runtime.
---
## Cross-standards (like grown-up APIs)
- **Auth:** `Authorization: Bearer sk-gw-...` (virtual gateway keys).
- **Errors:** OpenAI-compatible `{"error": {...}}` envelope — parsed natively by SDKs.
- **Routing metadata:** `X-Cortiq-*` headers on every response.
- **Cost accounting:** token usage + USD cost and the selected model.
- **Observability:** `GET /metrics` (Prometheus), `GET /healthz`, `GET /readyz`.
Details — [docs/PROTOCOLS.md](docs/PROTOCOLS.md).
---
## Build & test
```bash
cargo build --release # single self-contained binary (SPA embedded)
cargo test # config round-trip, routing validation
```
---
## Status
🚧 Implemented so far: OpenAI Chat Completions, routing with fallback & graceful
degradation, cost/token accounting, the **embedded multilingual admin console with
hot config reload**, statistics and Prometheus `GET /metrics`. Planned: streaming
(SSE), Anthropic provider/inbound, embeddings, MCP. See the roadmap in
[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). Contributions welcome — see
[CONTRIBUTING.md](CONTRIBUTING.md).
## License
Apache-2.0 — see [LICENSE](LICENSE).
README.ru.md
+177
View File
@@ -0,0 +1,177 @@
# Cortiq Gateway
[English](README.md) · **Русский**
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
![Rust](https://img.shields.io/badge/Rust-2021-orange.svg)
![Status](https://img.shields.io/badge/status-active-success.svg)
**Универсальный LLM-шлюз с интеллектуальной маршрутизацией.**
Один OpenAI-совместимый endpoint → автоматический выбор модели из вашего пула
(локальные дешёвые + хостовые дорогие) на основе типа задачи и сложности, которые
определяет [allaigate / cortiq-router](https://api.allaigate.com).
> Меняете в своём агенте/SDK только `base_url` — и получаете «умный» роутинг между
> моделями. Никакой логики выбора модели на стороне клиента.
```
┌─────────────┐ OpenAI / Anthropic / MCP ┌──────────────────┐
│ Агент / │ ─────────────────────────────▶│ Cortiq Gateway │
│ разработчик │ (стандартный протокол) │ (этот проект) │
└─────────────┘ ◀─────────────────────────────└──────────────────┘
ответ + метаданные │ │
│ /v1/route│ вызов LLM
▼ ▼
┌──────────────┐ ┌─────────────────────┐
│ cortiq-router│ │ Пул моделей │
│ (тип задачи, │ │ • local llama.cpp │
│ сложность) │ │ • ollama / vLLM │
└──────────────┘ │ • OpenAI / Claude │
└─────────────────────┘
```
---
## ✨ Ключевое
- **Drop-in OpenAI API.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`.
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен.
- **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема.
- **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`.
- **Playground.** Прогон промпта через живой пайплайн с разбором решения роутинга.
- **Один самодостаточный бинарь.** SPA встроена в Rust-бинарь — ничего лишнего разворачивать не нужно.
- **Секреты не покидают шлюз.** Агенты держат только виртуальный ключ шлюза; ключи провайдеров живут в шлюзе и наружу не уходят.
---
## 🖥️ Панель управления
![Дашборд](docs/screenshots/dashboard.png)
В шлюз встроена веб-панель на **`/admin`**:
| | |
|---|---|
| ![Модели](docs/screenshots/models.png) | ![Роутинг](docs/screenshots/routing.png) |
| **Models** — добавление/правка/probe моделей, ключи провайдеров | **Routing** — визуальный редактор полос (упорядоченный) |
| ![Playground](docs/screenshots/playground.png) | ![Дашборд](docs/screenshots/dashboard.png) |
| **Playground** — тест живого пайплайна с решением роутинга | **Dashboard** — трафик, стоимость, латентность |
```bash
cargo run --release -- --config config/gateway.toml --admin-token <ВАШ_ТОКЕН>
# откройте http://localhost:9000/admin?token=<ВАШ_ТОКЕН>
```
Если `--admin-token` / `[admin].token_env` не заданы — токен генерируется при старте и
печатается в лог. Все эндпоинты `/admin/api/*` требуют Bearer admin-токен; значения
секретов API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
---
## 🚀 Быстрый старт
```bash
# 1. поднимите роутер (хостовый allaigate или локальный cortiq-router)
# он слушает, например, http://localhost:8080 (или https://api.allaigate.com)
# 2. опишите свой пул моделей
cp config/gateway.example.toml config/gateway.toml
$EDITOR config/gateway.toml
# 3. запустите шлюз
cargo run --release -- --config config/gateway.toml
# Шлюз слушает 0.0.0.0:9000 и отдаёт панель на /admin
```
Теперь любой OpenAI-клиент работает через шлюз:
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:9000/v1", api_key="sk-gw-...")
resp = client.chat.completions.create(
model="cortiq-auto", # ← магическая модель = «выбери сам»
messages=[{"role": "user", "content": "Solve x^2 - 5x + 6 = 0"}],
)
print(resp.choices[0].message.content)
# заголовки ответа покажут выбор шлюза:
# X-Cortiq-Task-Label: math
# X-Cortiq-Complexity-Tier: low
# X-Cortiq-Selected-Model: local-qwen
```
`model: "cortiq-auto"` включает роутинг. Любое **реальное** имя модели из конфига
(`"gpt-4o-mini"`, `"local-qwen"`) — это passthrough напрямую, без роутинга.
> **Используете хостовый allaigate-роутер?** Укажите `url = "https://138.226.222.209"`,
> `verify_tls = false`, `taxonomy_id = "data-assistant"` и ключ `cortiq_…` в
> `CORTIQ_ROUTER_KEY`. На сложных запросах роутер эскалирует к oracle (~10 с), поэтому
> ставьте `timeout_ms = 12000+` — иначе шлюз мягко деградирует на default-модель.
---
## Поток запроса
1. Клиент → `POST /v1/chat/completions` (или Anthropic/MCP) с `model: "cortiq-auto"`.
2. Шлюз извлекает **текст для маршрутизации** (стратегия настраивается, см. [docs/ROUTING.ru.md](docs/ROUTING.ru.md)).
3. Шлюз → `cortiq-router /v1/route` → получает `task_label` + `complexity.tier`.
4. Шлюз выбирает модель из пула по таблице роутинга (с порядком fallback).
5. Шлюз → провайдер выбранной модели (транслируя протокол при необходимости), возвращает ответ.
6. Шлюз отдаёт метаданные роутинга в заголовках/`usage`, считает стоимость и статистику.
Подробно — [docs/ARCHITECTURE.ru.md](docs/ARCHITECTURE.ru.md).
---
## Поддерживаемые протоколы (входящие)
| Протокол | Endpoint | Статус |
|---|---|---|
| OpenAI Chat Completions | `POST /v1/chat/completions` | ✅ реализовано |
| OpenAI Completions (legacy) | `POST /v1/completions` | план |
| OpenAI Embeddings | `POST /v1/embeddings` | план |
| OpenAI Models | `GET /v1/models` | план |
| Anthropic Messages | `POST /v1/messages` | план |
| MCP (Model Context Protocol) | JSON-RPC / SSE | план |
| Native passthrough | `POST /route` | план |
## Поддерживаемые провайдеры (исходящие)
| Провайдер | Покрывает |
|---|---|
| `openai` (OpenAI-совместимый) | OpenAI, OpenRouter, Together, Groq, **vLLM, llama.cpp server, LM Studio, Ollama (`/v1`)** |
| `anthropic` | Claude Messages API (план) |
| `ollama` | нативный Ollama API |
| `http` | произвольный HTTP-эндпоинт (свой адаптер) |
---
## Конфигурация
Полный пример — [config/gateway.example.toml](config/gateway.example.toml).
Модель роутинга и cost-aware выбор — [docs/ROUTING.ru.md](docs/ROUTING.ru.md).
Всё из конфига также правится из панели управления в рантайме.
---
## Сборка и тесты
```bash
cargo build --release # один самодостаточный бинарь (SPA встроена)
cargo test # round-trip конфига, валидация роутинга
```
---
## Статус
🚧 Уже работает: OpenAI Chat Completions, роутинг с fallback и мягкой деградацией,
учёт стоимости/токенов, **встроенная многоязычная панель управления с горячей
перезагрузкой конфига**, статистика и Prometheus `GET /metrics`. В планах: стриминг
(SSE), провайдер/вход Anthropic, embeddings, MCP. Дорожная карта —
[docs/ARCHITECTURE.ru.md](docs/ARCHITECTURE.ru.md). Контрибьюции приветствуются —
см. [CONTRIBUTING.ru.md](CONTRIBUTING.ru.md).
## Лицензия
Apache-2.0 — см. [LICENSE](LICENSE).
config/gateway.example.toml
+152
View File
@@ -0,0 +1,152 @@
# ============================================================================
# Cortiq Gateway — example configuration
# Copy to gateway.toml and edit for your own model pool.
# Do NOT store secrets here — provide them via *_env (environment variables)
# or set them from the admin console (stored in config/secrets.toml).
# ============================================================================
listen = "0.0.0.0:9000"
# --- connection to cortiq-router (decides task type / complexity) -----------
# Hosted allaigate router: url = "https://138.226.222.209", verify_tls = false
# (cert is bound to an IP), taxonomy_id = "data-assistant", a cortiq_… key in
# CORTIQ_ROUTER_KEY.
# NOTE: on hard prompts the router escalates to an oracle LLM (~10 s) — for the
# hosted router set timeout_ms = 12000+, otherwise the gateway degrades to default.
[router]
url = "http://localhost:8080"
api_key_env = "CORTIQ_ROUTER_KEY" # name of the env var holding the router key
timeout_ms = 800 # timeout for the /v1/route call (12000+ for oracle)
taxonomy_id = "general.task-type" # for allaigate: "data-assistant"
verify_tls = true # false if the router uses an IP / self-signed cert
# --- how to pick the text to classify from the conversation -----------------
[route]
text_strategy = "last_user" # last_user | last_user_plus_system | concat_all
max_chars = 4000
cache_ttl = "60s"
profile = "balanced" # cost-saver | balanced | quality-first
# ============================================================================
# Model pool: mix cheap local models and expensive cloud ones.
# provider = "openai" covers OpenAI, OpenRouter, Together, Groq, and also
# LOCAL servers vLLM / llama.cpp / LM Studio / Ollama (/v1).
# ============================================================================
[[models]]
id = "local-qwen"
provider = "openai"
base_url = "http://localhost:8000/v1"
model = "qwen2.5-7b-instruct"
cost_tier = "cheap"
price_in = 0.0
price_out = 0.0
caps = ["tools"]
[[models]]
id = "gpt-4o-mini"
provider = "openai"
base_url = "https://api.openai.com/v1"
model = "gpt-4o-mini"
api_key_env = "OPENAI_API_KEY"
cost_tier = "mid"
price_in = 0.15
price_out = 0.60
caps = ["tools", "vision"]
[[models]]
id = "claude-opus"
provider = "anthropic"
base_url = "https://api.anthropic.com"
model = "claude-opus-4"
api_key_env = "ANTHROPIC_API_KEY"
cost_tier = "expensive"
price_in = 15.0
price_out = 75.0
caps = ["tools", "vision"]
# --- embedding model (for /v1/embeddings in provider mode) ------------------
[[models]]
id = "local-embed"
provider = "openai"
base_url = "http://localhost:8000/v1"
model = "nomic-embed-text"
cost_tier = "cheap"
kind = "embedding"
# ============================================================================
# Routing table: complexity tier → ordered pool (priority = fallback chain)
# ============================================================================
[routing]
low = ["local-qwen"]
medium = ["gpt-4o-mini", "local-qwen"]
high = ["claude-opus", "gpt-4o-mini"]
default = "gpt-4o-mini" # used when the router is unreachable
[routing.policy]
mode = "fixed_table" # fixed_table | cost_aware
# for cost_aware:
# min_class = { low = "cheap", medium = "mid", high = "expensive" }
# max_cost_usd_per_request = 0.50
# --- resilience -------------------------------------------------------------
[breaker]
threshold = 5
cooldown = "30s"
# ============================================================================
# Inbound protocols — enable only what you need (toggles apply live in the UI)
# ============================================================================
[protocols]
openai_chat = true
openai_completions = true
openai_embeddings = true
openai_models = true
anthropic_messages = false
mcp = false
native_passthrough = true
# --- response routing metadata ----------------------------------------------
[cortiq]
echo = true # add a "cortiq" field to the response body (X-Cortiq-* headers are always sent)
# ============================================================================
# Authentication: virtual gateway keys (agents get these, not provider keys).
# If this section is empty, the gateway runs in open mode (local development).
# ============================================================================
[[api_keys]]
key = "sk-gw-team-a"
account = "team-a"
rate_per_min = 600
# allow_models = ["cortiq-auto", "local-qwen"] # restrict available models (optional)
# --- idempotency and logging ------------------------------------------------
[idempotency]
ttl = "10m"
[log]
bodies = false # do NOT log message bodies (privacy) — default
level = "info"
[telemetry]
metrics = true # GET /metrics (Prometheus)
otlp_endpoint_env = "OTEL_EXPORTER_OTLP_ENDPOINT" # optional
# ============================================================================
# Admin web console — embedded in the binary, served at /admin
# ============================================================================
[admin]
enabled = true
# Name of the env var holding the admin token. If unset (or empty), a token is
# generated at startup and printed to the log (see also the --admin-token flag).
token_env = "CORTIQ_ADMIN_TOKEN"
# listen = "127.0.0.1:9001" # optional: separate bind for admin (default: main listen)
# ============================================================================
# Request statistics (dashboard + /metrics). Kept in memory + appended to JSONL.
# ============================================================================
[stats]
enabled = true
file = "config/stats.jsonl" # empty = in-memory only
ring_size = 500 # how many recent requests to keep in the feed
retention = "7d" # aggregate/series depth (replayed on startup)
docs/ARCHITECTURE.md
+205
View File
@@ -0,0 +1,205 @@
*Read this in [Русский](ARCHITECTURE.ru.md).*
# Cortiq Gateway Architecture
This document describes the internal design of the gateway, request flow, abstractions, and
roadmap. Related documents: [PROTOCOLS.md](PROTOCOLS.md) (protocol contracts),
[ROUTING.md](ROUTING.md) (model selection and cost model).
---
## 1. Purpose and Scope
**Cortiq Gateway** is an L7 proxy that:
1. accepts requests in one of the **standard protocols** (OpenAI, Anthropic, MCP);
2. asks **cortiq-router** what type of task it is and how complex;
3. selects a specific model from the **provider pool** (local + cloud);
4. calls the provider, **translating the protocol** when necessary;
5. returns the response to the client (streaming and non-streaming), adding routing metadata;
6. tracks cost and (optionally) closes the feedback loop back to the router.
What the gateway does **not** do: it does not store or train models (that is the router's
job), and it does not implement the LLMs themselves (that is the providers' job). The gateway
is thin, stateless (except for idempotency caches and rate-limit state), and horizontally
scalable.
---
## 2. Component Diagram
```
┌───────────────────────────────────────────────┐
│ Cortiq Gateway │
│ │
client (any protocol) │ ┌──────────────┐ ┌───────────────────┐ │
───────────────────────▶│──▶│ Inbound │──▶│ Core pipeline │ │
│ │ Protocol │ │ │ │
│ │ Adapters │ │ 1. extract text │ │
│ │ (openai/ │ │ 2. route() │────┼──▶ cortiq-router
│ │ anthropic/ │ │ 3. select model │ │ /v1/route
│ │ mcp) │ │ 4. translate │ │
│ └──────────────┘ │ 5. call provider │────┼──▶ Provider pool
│ │ 6. stream back │ │ (openai/anthropic/
│ ┌──────────────┐ │ 7. usage/cost │◀───┼── ollama/http)
│ │ Cross-cutting│ │ 8. feedback (opt)│────┼──▶ cortiq-router
│ │ auth · rate │ └───────────────────┘ │ /v1/feedback
│ │ idem · trace │ │
│ └──────────────┘ │
└───────────────────────────────────────────────┘
```
Code modules (`src/`):
| Module | Responsibility |
|---|---|
| `main.rs` | config loading, route registration for enabled protocols, server startup |
| `config.rs` | TOML parsing: model pool, routing table, protocols, keys |
| `router_client.rs` | client for `cortiq-router` (`/v1/route`, `/v1/feedback`) |
| `registry.rs` | model pool registry + `id → provider+endpoint` resolution |
| `routing.rs` | `(label, tier) → ordered model list`; fallback/default |
| `pipeline.rs` | core: extract → route → select → translate → call → stream |
| `providers/mod.rs` | `Provider` trait (unified LLM call interface) |
| `providers/openai.rs` | OpenAI-compatible adapter (covers vLLM/llama.cpp/ollama/openrouter) |
| `providers/anthropic.rs` | Anthropic Messages adapter |
| `protocols/mod.rs` | inbound adapter trait + shared types for the canonical message model |
| `protocols/openai_chat.rs` | `/v1/chat/completions` ↔ canonical model |
| `protocols/anthropic_messages.rs` | `/v1/messages` ↔ canonical model |
| `protocols/mcp.rs` | MCP server (JSON-RPC), routing exposed as a tool |
| `auth.rs` | virtual keys, rate limiting, quotas |
| `telemetry.rs` | Prometheus metrics, OpenTelemetry spans, request IDs |
| `error.rs` | unified error type + rendering into OpenAI error envelope |
---
## 3. Canonical Model (internal neutral format)
To avoid N incoming protocols × M providers turning into N×M translation paths,
everything is normalized internally to a **single canonical request/response model**.
```
ChatRequest {
routing: RoutingDirective, // auto | pinned(model_id) | profile
messages: Vec<Message>, // role + content (+ tool_calls)
tools: Vec<Tool>,
params: GenParams, // temperature, max_tokens, top_p, stop, ...
stream: bool,
meta: RequestMeta, // idempotency key, traceparent, account
}
ChatResponse {
id, model_used, choices, usage, finish_reason, cortiq: RouteInfo
}
```
- Inbound adapter: `protocol-specific → ChatRequest` and `ChatResponse → protocol-specific`.
- Outbound provider: `ChatRequest → provider wire` and `provider wire → ChatResponse`.
Adding a new protocol = one inbound adapter. Adding a new provider = one outbound adapter.
No combinatorial explosion.
---
## 4. Extension Traits
```rust
// inbound protocol
#[async_trait]
trait InboundProtocol {
fn routes(&self) -> Router; // axum routes for this protocol
fn name(&self) -> &'static str; // "openai_chat", ...
}
// outbound provider
#[async_trait]
trait Provider: Send + Sync {
fn id(&self) -> &str;
async fn chat(&self, req: ChatRequest) -> Result<ChatResponse>;
async fn chat_stream(&self, req: ChatRequest) -> Result<ChatStream>; // SSE stream
async fn embed(&self, req: EmbedRequest) -> Result<EmbedResponse>;
fn capabilities(&self) -> Caps; // tools? streaming? vision?
}
```
The registry (`registry.rs`) holds a `HashMap<model_id, Arc<dyn Provider>>` built
from `[[models]]`. The routing table (`routing.rs`) operates on `model_id`.
---
## 5. `route → select → call` Flow in Detail
1. **Extract.** Text for classification is taken from `messages`. Strategy from config:
`last_user` (default) | `concat_all` | `last_user_plus_system`. Long text is
truncated to `route.max_chars` (protection against cost/DoS).
2. **Route.** `router_client.route(text, profile)` → `{task_label, complexity}`.
Cached by text hash for `route.cache_ttl` (minor savings on repeated requests).
3. **Select.** `routing.select(label, tier)` → ordered `Vec<model_id>`.
Considers: model health, model `capabilities` (e.g. whether tools are needed),
and the cost policy (see [ROUTING.md](ROUTING.md)).
4. **Translate + Call.** Canonical `ChatRequest` → provider. Streaming is proxied
chunk-by-chunk (SSE → SSE) without buffering.
5. **Failover.** If a provider returns 5xx / timeout / overloaded — the next model
in the list is tried. The circuit breaker marks an unhealthy model to avoid
hammering it.
6. **Degrade.** If cortiq-router is unavailable, `routing.default` is used (soft
degradation — the request does not fail).
---
## 6. Reliability
| Mechanism | Behavior |
|---|---|
| Timeouts | separate timeouts for router call, provider connect, and provider stream |
| Failover | ordered model list within the tier |
| Circuit breaker | temporarily excludes a model after repeated errors |
| Router down | `routing.default` |
| All tier providers down | OpenAI error `503 upstream_unavailable`, `retriable=true` |
| Idempotency | repeat with the same `Idempotency-Key` returns the stored response |
---
## 7. Security and Multi-tenancy
- **Key isolation.** Agents hold only a virtual gateway key; provider keys
(`*_API_KEY`) live in the gateway environment and are never exposed externally.
- **Accounts.** Each `[[api_keys]]` entry maps to an account with its own rate limit,
quota, and optionally its own allowed-model subset and routing table.
- **PII.** Redaction is performed in the router before escalation to the oracle; the
gateway can additionally enable logging without request bodies (`log.bodies = false`
by default).
- **Audit.** Every request records: `request_id`, account, selected model, tokens, cost,
and decision path — without the prompt text by default.
---
## 8. Observability
`GET /metrics` (Prometheus), `GET /healthz`, `GET /readyz`. Metrics:
```
gw_requests_total{protocol,account}
gw_route_calls_total # calls to cortiq-router
gw_model_selected_total{model_id,tier}
gw_provider_calls_total{provider,model_id,outcome}
gw_failovers_total
gw_tokens_total{model_id,direction}
gw_cost_usd_total{account,model_id}
gw_latency_seconds{stage} # route | provider | total (histogram)
```
OpenTelemetry spans: `inbound → route → provider`, with `traceparent` forwarding.
---
## 9. Roadmap
| Phase | Contents |
|---|---|
| **v0.1** | OpenAI Chat/Completions/Embeddings/Models, `openai` provider, routing + fallback, auth, metrics, streaming |
| **v0.2** | `anthropic` provider + inbound Anthropic Messages, cost-aware selection, idempotency, circuit breaker |
| **v0.3** | MCP server, native `ollama` provider, OpenTelemetry, routing cache |
| **v0.4** | Per-account routing tables, feedback loop to router, load tests, Helm chart |
| **v1.0** | Contract stabilization, OpenAPI spec, official examples for crewAI/LangChain/Hermes |
Current phase: **Phase 0** (skeleton + specification in this repository).
docs/ARCHITECTURE.ru.md
+202
View File
@@ -0,0 +1,202 @@
*Read this in [English](ARCHITECTURE.md).*
# Архитектура Cortiq Gateway
Документ описывает внутреннее устройство шлюза, поток запроса, абстракции и
дорожную карту. Сопутствующие документы: [PROTOCOLS.md](PROTOCOLS.ru.md) (контракты
протоколов), [ROUTING.md](ROUTING.ru.md) (модель выбора моделей и стоимости).
---
## 1. Назначение и границы
**Cortiq Gateway** — это L7-прокси, который:
1. принимает запрос в одном из **стандартных протоколов** (OpenAI, Anthropic, MCP);
2. спрашивает **cortiq-router**, какой это тип задачи и насколько он сложный;
3. выбирает конкретную модель из **пула провайдеров** (локальные + облачные);
4. вызывает провайдера, **транслируя протокол** при необходимости;
5. возвращает ответ клиенту (стриминг и без), добавляя метаданные роутинга;
6. учитывает стоимость и (опц.) замыкает петлю обратной связи в роутер.
Что шлюз **не** делает: не хранит и не обучает модели (это роутер), не реализует
сами LLM (это провайдеры). Шлюз — тонкий, без состояния (кроме кэшей идемпотентности
и rate-limit), горизонтально масштабируется.
---
## 2. Компонентная схема
```
┌───────────────────────────────────────────────┐
│ Cortiq Gateway │
│ │
client (any protocol) │ ┌──────────────┐ ┌───────────────────┐ │
───────────────────────▶│──▶│ Inbound │──▶│ Core pipeline │ │
│ │ Protocol │ │ │ │
│ │ Adapters │ │ 1. extract text │ │
│ │ (openai/ │ │ 2. route() │────┼──▶ cortiq-router
│ │ anthropic/ │ │ 3. select model │ │ /v1/route
│ │ mcp) │ │ 4. translate │ │
│ └──────────────┘ │ 5. call provider │────┼──▶ Provider pool
│ │ 6. stream back │ │ (openai/anthropic/
│ ┌──────────────┐ │ 7. usage/cost │◀───┼── ollama/http)
│ │ Cross-cutting│ │ 8. feedback (opt)│────┼──▶ cortiq-router
│ │ auth · rate │ └───────────────────┘ │ /v1/feedback
│ │ idem · trace │ │
│ └──────────────┘ │
└───────────────────────────────────────────────┘
```
Модули кода (`src/`):
| Модуль | Ответственность |
|---|---|
| `main.rs` | загрузка конфига, сборка роутов по включённым протоколам, запуск сервера |
| `config.rs` | парсинг TOML: пул моделей, таблица роутинга, протоколы, ключи |
| `router_client.rs` | клиент к `cortiq-router` (`/v1/route`, `/v1/feedback`) |
| `registry.rs` | реестр моделей пула + резолв `id → провайдер+endpoint` |
| `routing.rs` | `(label, tier) → упорядоченный список моделей`; fallback/default |
| `pipeline.rs` | ядро: extract → route → select → translate → call → stream |
| `providers/mod.rs` | трейт `Provider` (унифицированный вызов LLM) |
| `providers/openai.rs` | OpenAI-совместимый адаптер (covers vLLM/llama.cpp/ollama/openrouter) |
| `providers/anthropic.rs` | адаптер Anthropic Messages |
| `protocols/mod.rs` | трейт входящего адаптера + общие типы канонической модели сообщения |
| `protocols/openai_chat.rs` | вход `/v1/chat/completions` ↔ каноническая модель |
| `protocols/anthropic_messages.rs` | вход `/v1/messages` ↔ каноническая модель |
| `protocols/mcp.rs` | MCP-сервер (JSON-RPC), маршрутизация как инструмент |
| `auth.rs` | виртуальные ключи, rate-limit, квоты |
| `telemetry.rs` | метрики Prometheus, OpenTelemetry-спаны, request-id |
| `error.rs` | единый тип ошибки + рендер в OpenAI-конверт |
---
## 3. Каноническая модель (внутренний нейтральный формат)
Чтобы N входящих протоколов × M провайдеров не превратились в N×M переводов,
внутри всё приводится к **одной канонической модели** запроса/ответа.
```
ChatRequest {
routing: RoutingDirective, // auto | pinned(model_id) | profile
messages: Vec<Message>, // role + content (+ tool_calls)
tools: Vec<Tool>,
params: GenParams, // temperature, max_tokens, top_p, stop, ...
stream: bool,
meta: RequestMeta, // idempotency key, traceparent, account
}
ChatResponse {
id, model_used, choices, usage, finish_reason, cortiq: RouteInfo
}
```
- Входящий адаптер: `protocol-specific → ChatRequest` и `ChatResponse → protocol-specific`.
- Исходящий провайдер: `ChatRequest → provider wire` и `provider wire → ChatResponse`.
Добавить новый протокол = один входящий адаптер. Добавить нового провайдера =
один исходящий адаптер. Никакого комбинаторного взрыва.
---
## 4. Трейты-расширения
```rust
// входящий протокол
#[async_trait]
trait InboundProtocol {
fn routes(&self) -> Router; // axum-роуты этого протокола
fn name(&self) -> &'static str; // "openai_chat", ...
}
// исходящий провайдер
#[async_trait]
trait Provider: Send + Sync {
fn id(&self) -> &str;
async fn chat(&self, req: ChatRequest) -> Result<ChatResponse>;
async fn chat_stream(&self, req: ChatRequest) -> Result<ChatStream>; // SSE-поток
async fn embed(&self, req: EmbedRequest) -> Result<EmbedResponse>;
fn capabilities(&self) -> Caps; // tools? streaming? vision?
}
```
Реестр (`registry.rs`) держит `HashMap<model_id, Arc<dyn Provider>>`, построенный
из `[[models]]`. Таблица роутинга (`routing.rs`) оперирует `model_id`.
---
## 5. Поток `route → select → call` детально
1. **Extract.** Из `messages` берётся текст для классификации. Стратегия из конфига:
`last_user` (по умолчанию) | `concat_all` | `last_user_plus_system`. Длинный
текст усечётся до `route.max_chars` (защита от стоимости/DoS).
2. **Route.** `router_client.route(text, profile)` → `{task_label, complexity}`.
Кэшируется по хэшу текста на `route.cache_ttl` (мелкая экономия на повторах).
3. **Select.** `routing.select(label, tier)` → упорядоченный `Vec<model_id>`.
Учитываются: доступность модели (health), её `capabilities` (нужны ли tools),
и cost-политика (см. [ROUTING.md](ROUTING.ru.md)).
4. **Translate + Call.** Канонический `ChatRequest` → провайдер. Стриминг проксируется
чанк-в-чанк (SSE → SSE) без буферизации.
5. **Failover.** Если провайдер вернул 5xx/таймаут/перегрузку — следующий из списка.
Circuit breaker помечает «больную» модель, чтобы не долбить её.
6. **Degrade.** Если cortiq-router недоступен — берётся `routing.default` (мягкая
деградация, запрос не падает).
---
## 6. Надёжность
| Механизм | Поведение |
|---|---|
| Таймауты | отдельные на router-вызов, connect и стрим провайдера |
| Failover | по упорядоченному списку моделей полосы |
| Circuit breaker | временно исключает модель после серии ошибок |
| Router down | `routing.default` |
| Все провайдеры полосы down | OpenAI-ошибка `503 upstream_unavailable`, `retriable=true` |
| Идемпотентность | повтор с тем же `Idempotency-Key` отдаёт сохранённый ответ |
---
## 7. Безопасность и мультиарендность
- **Изоляция ключей.** Агенты держат только виртуальный ключ шлюза; ключи провайдеров
(`*_API_KEY`) живут в окружении шлюза и наружу не уходят.
- **Аккаунты.** Каждый `[[api_keys]]` → аккаунт с rate-limit, квотой, (опц.) своим
подмножеством разрешённых моделей и своей таблицей роутинга.
- **PII.** Редакция выполняется в роутере перед эскалацией к оракулу; шлюз дополнительно
может включить логирование без тел сообщений (`log.bodies = false` по умолчанию).
- **Аудит.** Каждый запрос: `request_id`, аккаунт, выбранная модель, токены, стоимость,
путь решения — без текста промпта по умолчанию.
---
## 8. Наблюдаемость
`GET /metrics` (Prometheus), `GET /healthz`, `GET /readyz`. Метрики:
```
gw_requests_total{protocol,account}
gw_route_calls_total # обращения к cortiq-router
gw_model_selected_total{model_id,tier}
gw_provider_calls_total{provider,model_id,outcome}
gw_failovers_total
gw_tokens_total{model_id,direction}
gw_cost_usd_total{account,model_id}
gw_latency_seconds{stage} # route | provider | total (histogram)
```
OpenTelemetry-спаны: `inbound → route → provider`, с пробросом `traceparent`.
---
## 9. Дорожная карта
| Фаза | Содержимое |
|---|---|
| **v0.1** | OpenAI Chat/Completions/Embeddings/Models, провайдер `openai`, роутинг + fallback, auth, метрики, стриминг |
| **v0.2** | Провайдер `anthropic` + входящий Anthropic Messages, cost-aware выбор, идемпотентность, circuit breaker |
| **v0.3** | MCP-сервер, провайдер `ollama` нативный, OpenTelemetry, кэш роутинга |
| **v0.4** | Per-account таблицы роутинга, feedback-loop в роутер, нагрузочные тесты, Helm-чарт |
| **v1.0** | Стабилизация контракта, OpenAPI-спека, официальные примеры под crewAI/LangChain/Hermes |
Текущая фаза — **Phase 0** (скелет + спецификация в этом репозитории).
docs/PROTOCOLS.md
+324
View File
@@ -0,0 +1,324 @@
*Read this in [Русский](PROTOCOLS.ru.md).*
# Cortiq Gateway Protocols
Contracts for inbound protocols and cross-standard mechanisms. Each protocol is enabled
by a flag in `[protocols]`. Internally, all requests are normalized to the canonical model
(see [ARCHITECTURE.md](ARCHITECTURE.md#3-canonical-model-internal-neutral-format)).
Routing rule common to all protocols:
- `model = "cortiq-auto"` → enable intelligent routing via cortiq-router.
- `model = "cortiq-auto:quality-first"` → routing with the specified policy profile
(`cost-saver` | `balanced` | `quality-first`).
- `model = "<real id from config>"` → passthrough to that model without routing.
---
## Table of Contents
1. [OpenAI Chat Completions](#1-openai-chat-completions)
2. [OpenAI Completions (legacy)](#2-openai-completions-legacy)
3. [OpenAI Embeddings](#3-openai-embeddings)
4. [OpenAI Models](#4-openai-models)
5. [Anthropic Messages](#5-anthropic-messages)
6. [MCP (Model Context Protocol)](#6-mcp-model-context-protocol)
7. [Native passthrough](#7-native-passthrough)
8. [Cross-standard mechanisms](#8-cross-standard-mechanisms)
---
## 1. OpenAI Chat Completions
`POST /v1/chat/completions` — the primary endpoint; the de facto standard for agents.
### Request
```jsonc
{
"model": "cortiq-auto", // routing; or a real id for passthrough
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "Solve x^2 - 5x + 6 = 0" }
],
"temperature": 0.7,
"max_tokens": 1024,
"stream": false,
"tools": [ /* function-calling, proxied to the provider as-is */ ],
"tool_choice": "auto",
"stop": ["\n\n"],
"top_p": 1.0,
"response_format": { "type": "json_object" }
}
```
Standard OpenAI fields are supported; unknown fields are proxied to the provider unchanged
(forward-compatible).
### Response (non-streaming)
```jsonc
{
"id": "chatcmpl-9f...",
"object": "chat.completion",
"created": 1718800000,
"model": "local-qwen", // the ACTUALLY selected model
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "x = 2 or x = 3" },
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 24,
"completion_tokens": 11,
"total_tokens": 35
},
"cortiq": { // extension (enabled with cortiq.echo=true)
"task_label": "math",
"complexity": { "score": 0.22, "tier": "low" },
"selected_model": "local-qwen",
"route_source": "router", // router | cache | fallback | pinned
"router_request_id": "req_...",
"cost_usd": 0.0
}
}
```
> The `cortiq` field is an optional extension. OpenAI SDKs ignore it. If the client does
> not want extra fields, disable `cortiq.echo` — the metadata will remain in the response
> headers.
### Metadata in response headers (always present)
```
X-Cortiq-Request-Id: req_...
X-Cortiq-Task-Label: math
X-Cortiq-Complexity-Score: 0.22
X-Cortiq-Complexity-Tier: low
X-Cortiq-Selected-Model: local-qwen
X-Cortiq-Route-Source: router
X-Cortiq-Cost-Usd: 0.000000
```
### Response (streaming, `stream: true`)
SSE in OpenAI format: a series of `chat.completion.chunk` events, terminated by
`data: [DONE]`. The gateway proxies provider chunks one-to-one (without buffering),
adding routing metadata to HTTP headers before the stream begins.
```
data: {"id":"chatcmpl-9f","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"},"index":0}]}
data: {"id":"chatcmpl-9f","object":"chat.completion.chunk","choices":[{"delta":{"content":"x = 2"},"index":0}]}
data: {"id":"chatcmpl-9f","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop","index":0}]}
data: [DONE]
```
---
## 2. OpenAI Completions (legacy)
`POST /v1/completions` — for legacy integrations. Internally, `prompt` is wrapped into a
single user message and follows the same pipeline. Response `object` is `"text_completion"`.
```jsonc
// request
{ "model": "cortiq-auto", "prompt": "Translate to French: hello", "max_tokens": 64 }
// response
{ "object": "text_completion", "model": "local-qwen",
"choices": [ { "text": "bonjour", "index": 0, "finish_reason": "stop" } ],
"usage": { "prompt_tokens": 6, "completion_tokens": 2, "total_tokens": 8 } }
```
---
## 3. OpenAI Embeddings
`POST /v1/embeddings`. Two modes (configurable):
- `provider` — proxy to an embedding model from the pool;
- `router` — return an embedding from the **cortiq-router encoder** (the same vector the
router uses for classification — useful for consistent RAG).
```jsonc
// request
{ "model": "cortiq-embed", "input": ["first text", "second text"] }
// response
{
"object": "list",
"data": [
{ "object": "embedding", "index": 0, "embedding": [0.01, -0.02, /* ... */] },
{ "object": "embedding", "index": 1, "embedding": [0.03, 0.00, /* ... */] }
],
"model": "cortiq-embed",
"usage": { "prompt_tokens": 4, "total_tokens": 4 }
}
```
---
## 4. OpenAI Models
`GET /v1/models` — auto-discovery for clients. Returns the gateway's virtual models
(`cortiq-auto`, `cortiq-embed`) and all real models in the pool.
```jsonc
{
"object": "list",
"data": [
{ "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "local-qwen", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
]
}
```
---
## 5. Anthropic Messages
`POST /v1/messages` — for Claude-native agents and the Anthropic SDK.
```jsonc
// request
{
"model": "cortiq-auto",
"max_tokens": 1024,
"system": "You are a helpful assistant.",
"messages": [ { "role": "user", "content": "Solve x^2 - 5x + 6 = 0" } ],
"stream": false
}
// response
{
"id": "msg_...",
"type": "message",
"role": "assistant",
"model": "local-qwen",
"content": [ { "type": "text", "text": "x = 2 or x = 3" } ],
"stop_reason": "end_turn",
"usage": { "input_tokens": 24, "output_tokens": 11 }
}
```
Streaming uses native Anthropic events (`message_start`, `content_block_delta`,
`message_delta`, `message_stop`). If the selected model is a non-Anthropic provider,
the gateway translates through the canonical model in both directions.
---
## 6. MCP (Model Context Protocol)
The gateway runs an **MCP server** (JSON-RPC 2.0 over HTTP+SSE or stdio) and exposes
routing as tools for MCP-native orchestrators.
Published tools:
| Tool | Purpose |
|---|---|
| `route_and_complete` | classify + execute on the selected model, return the response |
| `classify_task` | classification only (task type + complexity), no execution |
`tools/list` returns JSON schemas. Example `tools/call` invocation:
```jsonc
// request (JSON-RPC)
{
"jsonrpc": "2.0", "id": 1, "method": "tools/call",
"params": {
"name": "route_and_complete",
"arguments": { "prompt": "Solve x^2 - 5x + 6 = 0", "profile": "balanced" }
}
}
// response
{
"jsonrpc": "2.0", "id": 1,
"result": {
"content": [ { "type": "text", "text": "x = 2 or x = 3" } ],
"isError": false,
"_meta": { "task_label": "math", "tier": "low", "selected_model": "local-qwen" }
}
}
```
---
## 7. Native passthrough
`POST /route` — a thin wrapper over `cortiq-router /v1/route` for clients that need only
the **routing decision** (without execution). The request body and response match the router
contract (see `cortiq-router/docs/ФОРМАТ_ЗАПРОСА_И_ОТВЕТА.md`). Useful for debugging and
for integrators who want to manage the LLM call themselves.
---
## 8. Cross-standard mechanisms
Common to all HTTP protocols.
### 8.1 Authentication
```
Authorization: Bearer sk-gw-...
```
Virtual gateway key. Mapped to an account (`[[api_keys]]`). If no keys are configured,
the gateway runs in open mode (for local development). Provider keys are never exposed.
### 8.2 Idempotency
```
Idempotency-Key: <unique-operation-id>
```
A repeat request with the same key within the `idempotency.ttl` window returns the
**stored** response (safe agent retries, without double-invoking the LLM or double billing).
### 8.3 Rate limiting
On `429`, the following headers are returned:
```
Retry-After: 2
X-RateLimit-Limit: 600
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1718800060
```
### 8.4 Tracing
W3C `traceparent` is accepted and forwarded; OpenTelemetry spans cover
`inbound→route→provider`. The response includes `X-Request-Id`. This enables an
end-to-end trace from agent → gateway → provider.
### 8.5 Error format (OpenAI-compatible)
```jsonc
{
"error": {
"message": "no healthy model available for tier 'high'",
"type": "upstream_unavailable", // invalid_request_error | authentication_error | rate_limit_error | upstream_unavailable | internal_error
"code": "no_healthy_model",
"param": null
}
}
```
OpenAI/Anthropic SDKs parse this envelope natively. Status code mapping:
| Type | HTTP | retriable |
|---|---|---|
| `invalid_request_error` | 400 | no |
| `authentication_error` | 401 | no |
| `rate_limit_error` | 429 | yes |
| `upstream_unavailable` | 502/503 | yes |
| `internal_error` | 500 | yes |
### 8.6 Cost tracking
`usage` is returned in the protocol format (OpenAI/Anthropic). Additionally, cost is
included in the `X-Cortiq-Cost-Usd` header and in `cortiq.cost_usd` (if `cortiq.echo`
is enabled). The price is calculated from the model's `price_in`/`price_out` in the config
multiplied by the token counts.
docs/PROTOCOLS.ru.md
+321
View File
@@ -0,0 +1,321 @@
*Read this in [English](PROTOCOLS.md).*
# Протоколы Cortiq Gateway
Контракты входящих протоколов и кросс-стандартных механизмов. Каждый протокол
включается флагом в `[protocols]`. Внутри всё приводится к канонической модели
(см. [ARCHITECTURE.md](ARCHITECTURE.ru.md#3-каноническая-модель)).
Общее правило маршрутизации во всех протоколах:
- `model = "cortiq-auto"` → включить интеллектуальный роутинг через cortiq-router.
- `model = "cortiq-auto:quality-first"` → роутинг с указанным policy-профилем
(`cost-saver` | `balanced` | `quality-first`).
- `model = "<реальный id из конфига>"` → passthrough к этой модели без роутинга.
---
## Оглавление
1. [OpenAI Chat Completions](#1-openai-chat-completions)
2. [OpenAI Completions (legacy)](#2-openai-completions-legacy)
3. [OpenAI Embeddings](#3-openai-embeddings)
4. [OpenAI Models](#4-openai-models)
5. [Anthropic Messages](#5-anthropic-messages)
6. [MCP (Model Context Protocol)](#6-mcp-model-context-protocol)
7. [Native passthrough](#7-native-passthrough)
8. [Кросс-стандарты](#8-кросс-стандарты)
---
## 1. OpenAI Chat Completions
`POST /v1/chat/completions` — основной endpoint, де-факто стандарт для агентов.
### Запрос
```jsonc
{
"model": "cortiq-auto", // роутинг; или реальный id для passthrough
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "Solve x^2 - 5x + 6 = 0" }
],
"temperature": 0.7,
"max_tokens": 1024,
"stream": false,
"tools": [ /* function-calling, проксируется к провайдеру как есть */ ],
"tool_choice": "auto",
"stop": ["\n\n"],
"top_p": 1.0,
"response_format": { "type": "json_object" }
}
```
Поддерживаются стандартные поля OpenAI; неизвестные поля проксируются к провайдеру
без изменений (forward-compatible).
### Ответ (без стриминга)
```jsonc
{
"id": "chatcmpl-9f...",
"object": "chat.completion",
"created": 1718800000,
"model": "local-qwen", // РЕАЛЬНО выбранная модель
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "x = 2 или x = 3" },
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 24,
"completion_tokens": 11,
"total_tokens": 35
},
"cortiq": { // расширение (включается cortiq.echo=true)
"task_label": "math",
"complexity": { "score": 0.22, "tier": "low" },
"selected_model": "local-qwen",
"route_source": "router", // router | cache | fallback | pinned
"router_request_id": "req_...",
"cost_usd": 0.0
}
}
```
> Поле `cortiq` — необязательное расширение. OpenAI-SDK его игнорируют. Если клиент
> не хочет «лишних» полей — выключите `cortiq.echo`, метаданные останутся в заголовках.
### Метаданные в заголовках (всегда)
```
X-Cortiq-Request-Id: req_...
X-Cortiq-Task-Label: math
X-Cortiq-Complexity-Score: 0.22
X-Cortiq-Complexity-Tier: low
X-Cortiq-Selected-Model: local-qwen
X-Cortiq-Route-Source: router
X-Cortiq-Cost-Usd: 0.000000
```
### Ответ (стриминг, `stream: true`)
SSE, формат OpenAI: серия чанков `chat.completion.chunk`, завершается `data: [DONE]`.
Шлюз проксирует чанки провайдера один-в-один (без буферизации), добавив метаданные
роутинга в HTTP-заголовки до начала потока.
```
data: {"id":"chatcmpl-9f","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"},"index":0}]}
data: {"id":"chatcmpl-9f","object":"chat.completion.chunk","choices":[{"delta":{"content":"x = 2"},"index":0}]}
data: {"id":"chatcmpl-9f","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop","index":0}]}
data: [DONE]
```
---
## 2. OpenAI Completions (legacy)
`POST /v1/completions` — для старых интеграций. Внутри `prompt` оборачивается в
одно user-сообщение и идёт по тому же пути. Ответ — `object: "text_completion"`.
```jsonc
// запрос
{ "model": "cortiq-auto", "prompt": "Translate to French: hello", "max_tokens": 64 }
// ответ
{ "object": "text_completion", "model": "local-qwen",
"choices": [ { "text": "bonjour", "index": 0, "finish_reason": "stop" } ],
"usage": { "prompt_tokens": 6, "completion_tokens": 2, "total_tokens": 8 } }
```
---
## 3. OpenAI Embeddings
`POST /v1/embeddings`. Два режима (настраивается):
- `provider` — проксировать к embedding-модели из пула;
- `router` — вернуть эмбеддинг **энкодером cortiq-router** (тот же вектор, что роутер
использует для классификации — удобно для согласованного RAG).
```jsonc
// запрос
{ "model": "cortiq-embed", "input": ["first text", "second text"] }
// ответ
{
"object": "list",
"data": [
{ "object": "embedding", "index": 0, "embedding": [0.01, -0.02, /* ... */] },
{ "object": "embedding", "index": 1, "embedding": [0.03, 0.00, /* ... */] }
],
"model": "cortiq-embed",
"usage": { "prompt_tokens": 4, "total_tokens": 4 }
}
```
---
## 4. OpenAI Models
`GET /v1/models` — автодискавери для клиентов. Возвращает виртуальные модели шлюза
(`cortiq-auto`, `cortiq-embed`) и все реальные модели пула.
```jsonc
{
"object": "list",
"data": [
{ "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "local-qwen", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
]
}
```
---
## 5. Anthropic Messages
`POST /v1/messages` — для Claude-нативных агентов и Anthropic SDK.
```jsonc
// запрос
{
"model": "cortiq-auto",
"max_tokens": 1024,
"system": "You are a helpful assistant.",
"messages": [ { "role": "user", "content": "Solve x^2 - 5x + 6 = 0" } ],
"stream": false
}
// ответ
{
"id": "msg_...",
"type": "message",
"role": "assistant",
"model": "local-qwen",
"content": [ { "type": "text", "text": "x = 2 или x = 3" } ],
"stop_reason": "end_turn",
"usage": { "input_tokens": 24, "output_tokens": 11 }
}
```
Стриминг — нативные Anthropic-события (`message_start`, `content_block_delta`,
`message_delta`, `message_stop`). Если выбранная модель — не-Anthropic-провайдер,
шлюз транслирует через каноническую модель в обе стороны.
---
## 6. MCP (Model Context Protocol)
Шлюз поднимает **MCP-сервер** (JSON-RPC 2.0 поверх HTTP+SSE или stdio) и публикует
маршрутизацию как инструменты для MCP-нативных оркестраторов.
Публикуемые инструменты:
| Инструмент | Назначение |
|---|---|
| `route_and_complete` | классифицировать + выполнить на выбранной модели, вернуть ответ |
| `classify_task` | только классификация (тип задачи + сложность), без выполнения |
`tools/list` отдаёт JSON-схемы. Пример вызова `tools/call`:
```jsonc
// запрос (JSON-RPC)
{
"jsonrpc": "2.0", "id": 1, "method": "tools/call",
"params": {
"name": "route_and_complete",
"arguments": { "prompt": "Solve x^2 - 5x + 6 = 0", "profile": "balanced" }
}
}
// ответ
{
"jsonrpc": "2.0", "id": 1,
"result": {
"content": [ { "type": "text", "text": "x = 2 или x = 3" } ],
"isError": false,
"_meta": { "task_label": "math", "tier": "low", "selected_model": "local-qwen" }
}
}
```
---
## 7. Native passthrough
`POST /route` — тонкая обёртка над `cortiq-router /v1/route` для клиентов, которым
нужно только **решение** (без выполнения). Тело и ответ совпадают с контрактом роутера
(см. `cortiq-router/docs/ФОРМАТ_ЗАПРОСА_И_ОТВЕТА.md`). Полезно для отладки и для тех,
кто хочет сам управлять вызовом LLM.
---
## 8. Кросс-стандарты
Едины для всех HTTP-протоколов.
### 8.1 Аутентификация
```
Authorization: Bearer sk-gw-...
```
Виртуальный ключ шлюза. Маппится на аккаунт (`[[api_keys]]`). Если ключи не настроены —
открытый режим (для локальной разработки). Ключи провайдеров наружу не отдаются.
### 8.2 Идемпотентность
```
Idempotency-Key: <уникальный-id-операции>
```
Повтор с тем же ключом в окне `idempotency.ttl` отдаёт **сохранённый** ответ
(безопасные ретраи агентов, без двойного вызова LLM и двойного биллинга).
### 8.3 Rate-limit
На `429` возвращаются:
```
Retry-After: 2
X-RateLimit-Limit: 600
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1718800060
```
### 8.4 Трассировка
Приём и проброс W3C `traceparent`; OpenTelemetry-спаны `inbound→route→provider`.
В ответе — `X-Request-Id`. Это позволяет сшить сквозной трейс агент → шлюз → провайдер.
### 8.5 Формат ошибки (OpenAI-совместимый)
```jsonc
{
"error": {
"message": "no healthy model available for tier 'high'",
"type": "upstream_unavailable", // invalid_request_error | authentication_error | rate_limit_error | upstream_unavailable | internal_error
"code": "no_healthy_model",
"param": null
}
}
```
OpenAI/Anthropic SDK парсят такой конверт нативно. Сопоставление статусов:
| Тип | HTTP | retriable |
|---|---|---|
| `invalid_request_error` | 400 | нет |
| `authentication_error` | 401 | нет |
| `rate_limit_error` | 429 | да |
| `upstream_unavailable` | 502/503 | да |
| `internal_error` | 500 | да |
### 8.6 Учёт стоимости
`usage` отдаётся в формате протокола (OpenAI/Anthropic). Дополнительно — стоимость
в заголовке `X-Cortiq-Cost-Usd` и в `cortiq.cost_usd` (если включено `cortiq.echo`).
Цена считается из `price_in`/`price_out` модели в конфиге × токены.
docs/ROUTING.md
+164
View File
@@ -0,0 +1,164 @@
*Read this in [Русский](ROUTING.ru.md).*
# Routing and Cost
How Cortiq Gateway translates a router decision (`task_label` + `complexity.tier`) into
selection of a specific model from your pool — and how to make this **cost-aware**:
cheap tasks stay local, expensive ones go to the cloud.
---
## 1. Base model: complexity tier → model pool
cortiq-router returns a complexity tier (`low` / `medium` / `high`, configurable in the
router). The `[routing]` table maps each tier to an **ordered list** of models — which
serves simultaneously as a priority order and a fallback chain.
```toml
[routing]
low = ["local-qwen"] # simple → local only (free)
medium = ["gpt-4o-mini", "local-qwen"] # medium → cheap hosted, fall back to local
high = ["claude-opus", "gpt-4o-mini"] # complex → expensive model, fall back to mid
default = "gpt-4o-mini" # if the router is unavailable
```
Selection algorithm:
```
1. tier := router.complexity.tier (or the default tier if router is unavailable)
2. candidates := routing[tier] (ordered list of model_id)
3. for each model_id in order:
if the model is healthy AND supports the required capabilities (tools/vision):
select it → done
4. if all are unavailable: try the next tier down/up per policy, or return error 503
```
---
## 2. Universal pool: cheap local + expensive hosted
The primary developer scenario: mix a **free local** model with a **paid cloud** API,
and let the gateway decide which one to use.
```toml
# Local model on vLLM / llama.cpp / Ollama (OpenAI-compatible endpoint)
[[models]]
id = "local-qwen"
provider = "openai"
base_url = "http://localhost:8000/v1"
model = "qwen2.5-7b-instruct"
cost_tier = "cheap"
price_in = 0.0 # local → 0
price_out = 0.0
caps = ["tools"] # capabilities (checked during selection)
# Cheap cloud
[[models]]
id = "gpt-4o-mini"
provider = "openai"
base_url = "https://api.openai.com/v1"
model = "gpt-4o-mini"
api_key_env = "OPENAI_API_KEY"
cost_tier = "mid"
price_in = 0.15 # $/1M input tokens
price_out = 0.60
# Expensive cloud for complex tasks
[[models]]
id = "claude-opus"
provider = "anthropic"
base_url = "https://api.anthropic.com"
model = "claude-opus-4"
api_key_env = "ANTHROPIC_API_KEY"
cost_tier = "expensive"
price_in = 15.0
price_out = 75.0
caps = ["tools", "vision"]
```
Because both `local-qwen` and `gpt-4o-mini` use `provider = "openai"`, the gateway does
not care where the model is physically located — **local servers already speak the OpenAI
API**. This is what makes the proxy truly universal.
---
## 3. Cost-aware mode (optional)
Instead of a fixed table, you can enable selection by **budget/price** (v0.2):
```toml
[routing.policy]
mode = "cost_aware" # fixed_table | cost_aware
# in cost_aware mode the gateway picks the cheapest model for each tier
# that meets the minimum quality class for that tier:
min_class = { low = "cheap", medium = "mid", high = "expensive" }
# upper price bound per request (guard against expensive calls)
max_cost_usd_per_request = 0.50
```
Logic: among models that are at least `min_class[tier]` and within the price cap, the
cheapest available model is selected. This yields the "minimally sufficient" model for
the task complexity.
---
## 4. Pinning and passthrough
- **Passthrough.** `model = "local-qwen"` (a real id) — no router call; direct invocation.
Useful when the client already knows what it wants.
- **Profile pinning.** `model = "cortiq-auto:cost-saver"` — routing with a specific router
policy profile (escalates less often → cheaper).
- **Router version pinning** (v0.4). To prevent routing behavior from drifting due to
router self-learning, you can pin `model_version` — the agent gets reproducible routing.
---
## 5. Text extraction for routing (multi-turn)
An agent request is not a single string but a system prompt + history + tools. What to
send to the router for classification is configurable:
```toml
[route]
text_strategy = "last_user" # last_user | concat_all | last_user_plus_system
max_chars = 4000 # truncate long context (cost/DoS protection)
cache_ttl = "60s" # cache routing decisions by text hash
profile = "balanced" # default policy profile
```
| Strategy | What is classified | When to use |
|---|---|---|
| `last_user` | last user message | default; cheap and accurate for most agents |
| `last_user_plus_system` | system prompt + last user message | when the role is set by the system prompt |
| `concat_all` | full conversation (truncated) | multi-turn with meaning spread across history |
---
## 6. Failover and circuit breaker
- **Failover.** On 5xx / timeout / `overloaded` from a provider — try the next model in
the list.
- **Circuit breaker.** After `breaker.threshold` consecutive errors, the model is marked
"open" for `breaker.cooldown`; requests skip it. Half-open: one probe request after the
cooldown expires.
- **Router down.** `routing.default` is used; the request does not fail (soft degradation).
```toml
[breaker]
threshold = 5
cooldown = "30s"
```
---
## 7. Feedback loop (optional, v0.4)
When enabled, the gateway can send a `/v1/feedback` signal to the router after a successful
completion (e.g. based on the result of agent-side response validation) — the router will
fine-tune the task mask. Over time this makes routing fit your specific pool more precisely.
```toml
[feedback]
enabled = false
# the ground-truth source is set by the integrator (e.g. the X-Cortiq-Correct-Label header)
```
docs/ROUTING.ru.md
+162
View File
@@ -0,0 +1,162 @@
*Read this in [English](ROUTING.md).*
# Маршрутизация и стоимость
Как Cortiq Gateway превращает решение роутера (`task_label` + `complexity.tier`) в
выбор конкретной модели из вашего пула — и как сделать это **cost-aware**:
дешёвое локально, дорогое в облаке.
---
## 1. Базовая модель: полоса сложности → пул моделей
cortiq-router возвращает полосу сложности (`low` / `medium` / `high`, настраивается
в роутере). Таблица `[routing]` сопоставляет полосе **упорядоченный список** моделей —
это одновременно приоритет и цепочка fallback.
```toml
[routing]
low = ["local-qwen"] # простое → только локально (бесплатно)
medium = ["gpt-4o-mini", "local-qwen"] # средне → дешёвый хост, откат на локальное
high = ["claude-opus", "gpt-4o-mini"] # сложно → дорогая модель, откат на средн.
default = "gpt-4o-mini" # если роутер недоступен
```
Алгоритм выбора:
```
1. tier := router.complexity.tier (или default-полоса, если роутер недоступен)
2. candidates := routing[tier] (упорядоченный список model_id)
3. для каждого model_id по порядку:
если модель healthy И поддерживает нужные возможности (tools/vision):
выбрать её → выход
4. если все недоступны: следующая полоса вниз/вверх по политике, либо ошибка 503
```
---
## 2. Универсальный пул: локальные дешёвые + хостовые дорогие
Главный сценарий для разработчика: смешать **бесплатную локальную** модель и
**платный облачный** API, и пусть шлюз сам решает, что куда.
```toml
# Локальная модель на vLLM / llama.cpp / Ollama (OpenAI-совместимый endpoint)
[[models]]
id = "local-qwen"
provider = "openai"
base_url = "http://localhost:8000/v1"
model = "qwen2.5-7b-instruct"
cost_tier = "cheap"
price_in = 0.0 # локально → 0
price_out = 0.0
caps = ["tools"] # что умеет (для проверки на этапе выбора)
# Дешёвый облачный
[[models]]
id = "gpt-4o-mini"
provider = "openai"
base_url = "https://api.openai.com/v1"
model = "gpt-4o-mini"
api_key_env = "OPENAI_API_KEY"
cost_tier = "mid"
price_in = 0.15 # $/1M входных токенов
price_out = 0.60
# Дорогой облачный для сложного
[[models]]
id = "claude-opus"
provider = "anthropic"
base_url = "https://api.anthropic.com"
model = "claude-opus-4"
api_key_env = "ANTHROPIC_API_KEY"
cost_tier = "expensive"
price_in = 15.0
price_out = 75.0
caps = ["tools", "vision"]
```
Поскольку `local-qwen` и `gpt-4o-mini` оба `provider = "openai"`, шлюзу не важно,
где модель физически — **локальные серверы уже говорят на OpenAI API**. Это и делает
прокси универсальным.
---
## 3. Cost-aware режим (опционально)
Вместо жёсткой таблицы можно включить выбор по **бюджету/цене** (v0.2):
```toml
[routing.policy]
mode = "cost_aware" # fixed_table | cost_aware
# при cost_aware шлюз для каждой полосы берёт самую дешёвую модель,
# удовлетворяющую минимальному «классу качества» полосы:
min_class = { low = "cheap", medium = "mid", high = "expensive" }
# верхняя граница цены за запрос (страховка от дорогих вызовов)
max_cost_usd_per_request = 0.50
```
Логика: среди моделей не ниже `min_class[tier]` и не дороже лимита выбирается самая
дешёвая доступная. Это даёт «минимально достаточную» модель под сложность задачи.
---
## 4. Pinning и passthrough
- **Passthrough.** `model = "local-qwen"` (реальный id) — без обращения к роутеру,
прямой вызов. Полезно, когда клиент сам знает, что хочет.
- **Pinning профиля.** `model = "cortiq-auto:cost-saver"` — роутинг с конкретным
policy-профилем роутера (реже эскалирует → дешевле).
- **Pinning версии роутера** (v0.4). Чтобы поведение не «плыло» из-за самообучения
роутера, можно зафиксировать `model_version` — агент получает воспроизводимый роутинг.
---
## 5. Извлечение текста для маршрутизации (мультитёрн)
Агентский запрос — это не одна строка, а system + история + tools. Что слать роутеру
на классификацию, настраивается:
```toml
[route]
text_strategy = "last_user" # last_user | concat_all | last_user_plus_system
max_chars = 4000 # усечение длинного контекста (стоимость/DoS)
cache_ttl = "60s" # кэш решения по хэшу текста
profile = "balanced" # дефолтный policy-профиль
```
| Стратегия | Что классифицируем | Когда |
|---|---|---|
| `last_user` | последнее сообщение пользователя | по умолчанию; дёшево и точно для большинства агентов |
| `last_user_plus_system` | system + последнее user | когда роль задаётся системным промптом |
| `concat_all` | весь диалог (усечённый) | мультитёрн со смыслом, размазанным по истории |
---
## 6. Failover и circuit breaker
- **Failover.** При 5xx/таймауте/`overloaded` от провайдера — следующая модель списка.
- **Circuit breaker.** После `breaker.threshold` ошибок подряд модель помечается
«открытой» на `breaker.cooldown`, запросы её минуют. Полупроброс — один пробный
запрос после кулдауна.
- **Router down.** Берётся `routing.default`, запрос не падает (мягкая деградация).
```toml
[breaker]
threshold = 5
cooldown = "30s"
```
---
## 7. Петля обратной связи (опционально, v0.4)
Если включено, шлюз после успешного выполнения может отправить в роутер
`/v1/feedback` сигнал (например, по результату валидации ответа агентом) — роутер
дообучит маску задачи. Так маршрутизация со временем точнее ложится на ваш пул.
```toml
[feedback]
enabled = false
# источник истины задаёт интегратор (например, заголовок X-Cortiq-Correct-Label)
```
docs/screenshots/dashboard.png
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 230 KiB

docs/screenshots/models.png
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 146 KiB

docs/screenshots/playground.png
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 137 KiB

docs/screenshots/routing.png
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 140 KiB

src/admin/assets.rs
+58
View File
@@ -0,0 +1,58 @@
//! Embedded static assets for the admin panel (SPA). The `web/` directory is compiled
//! into the binary via `rust-embed`, so the gateway remains a single self-contained file.
//!
//! Served as the main router's fallback: everything under `/admin/*` not matched by
//! an explicit route is served as a file; an unknown path under `/admin` falls back to
//! `index.html` (the client-side hash router handles navigation).
use axum::{
http::{header, StatusCode, Uri},
response::{IntoResponse, Response},
};
use rust_embed::RustEmbed;
#[derive(RustEmbed)]
#[folder = "web/"]
struct WebAssets;
fn serve_file(path: &str) -> Response {
match WebAssets::get(path) {
Some(content) => {
let mime = mime_guess::from_path(path).first_or_octet_stream();
(
[(header::CONTENT_TYPE, mime.as_ref())],
content.data.into_owned(),
)
.into_response()
}
None => {
// SPA: unknown path under /admin → serve index.html
match WebAssets::get("index.html") {
Some(content) => (
[(header::CONTENT_TYPE, "text/html; charset=utf-8")],
content.data.into_owned(),
)
.into_response(),
None => (StatusCode::NOT_FOUND, "not found").into_response(),
}
}
}
}
/// Application-wide fallback: serves `/admin/*`; everything else → 404 JSON.
pub async fn fallback(uri: Uri) -> Response {
let path = uri.path();
if let Some(rest) = path.strip_prefix("/admin") {
let rest = rest.trim_start_matches('/');
let file = if rest.is_empty() { "index.html" } else { rest };
serve_file(file)
} else {
(
StatusCode::NOT_FOUND,
axum::Json(serde_json::json!({
"error": { "message": "not found", "type": "not_found", "code": "not_found" }
})),
)
.into_response()
}
}
src/admin/mod.rs
+756
View File
@@ -0,0 +1,756 @@
//! Admin API for the web management panel (`/admin/api/*`) + SPA serving (`/admin/*`).
//!
//! All endpoints under `/admin/api` are protected by a Bearer admin token. Config changes
//! go through `AppState::reload` (validation → build `Live` → atomic TOML write
//! → swap), so they are applied without a restart. Secrets are never returned —
//! only their presence status.
pub mod assets;
use crate::config::{
AdminCfg, ApiKeyCfg, BreakerCfg, Config, CortiqCfg, LogCfg, ModelCfg, ProtocolsCfg, RouteCfg,
RouterCfg, RoutingCfg, RoutingPolicy, StatsCfg, TelemetryCfg, TierTargets,
};
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
use crate::state::SharedState;
use crate::stats::parse_duration_secs;
use axum::{
extract::{Path, Query, Request, State},
http::{header, StatusCode},
middleware::{self, Next},
response::{IntoResponse, Response},
routing::{get, post},
Json, Router,
};
use serde::Deserialize;
use serde_json::{json, Value};
use std::collections::HashMap;
use std::time::Instant;
// ───────────────────────────── common types / helpers ───────────────────────
type ApiResult = Result<Json<Value>, ApiError>;
pub struct ApiError {
status: StatusCode,
message: String,
}
impl ApiError {
fn bad(msg: impl Into<String>) -> Self {
Self {
status: StatusCode::BAD_REQUEST,
message: msg.into(),
}
}
fn not_found(msg: impl Into<String>) -> Self {
Self {
status: StatusCode::NOT_FOUND,
message: msg.into(),
}
}
}
impl From<anyhow::Error> for ApiError {
fn from(e: anyhow::Error) -> Self {
Self::bad(e.to_string())
}
}
impl IntoResponse for ApiError {
fn into_response(self) -> Response {
(
self.status,
Json(json!({"error": {"message": self.message, "type": "admin_error"}})),
)
.into_response()
}
}
fn ok(v: Value) -> ApiResult {
Ok(Json(v))
}
fn current_cfg(state: &SharedState) -> Config {
state.live().cfg.clone()
}
/// Random hex token from `n_bytes` bytes of cryptographic randomness.
pub fn random_token(n_bytes: usize) -> String {
let mut buf = vec![0u8; n_bytes];
if let Ok(mut f) = std::fs::File::open("/dev/urandom") {
use std::io::Read;
let _ = f.read_exact(&mut buf);
} else {
let t = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_nanos();
for (i, b) in buf.iter_mut().enumerate() {
*b = ((t >> ((i % 16) * 8)) as u8) ^ (i as u8).wrapping_mul(31);
}
}
buf.iter().map(|b| format!("{b:02x}")).collect()
}
fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
if a.len() != b.len() {
return false;
}
let mut diff = 0u8;
for (x, y) in a.iter().zip(b.iter()) {
diff |= x ^ y;
}
diff == 0
}
fn mask_key(key: &str) -> String {
let n = key.chars().count();
if n <= 8 {
return "•".repeat(n.max(3));
}
let head: String = key.chars().take(6).collect();
let tail: String = key.chars().skip(n - 4).collect();
format!("{head}…{tail}")
}
fn tiers_to_map(routing: &RoutingCfg) -> HashMap<String, Vec<String>> {
routing
.tiers
.iter()
.map(|(k, v)| {
let list = match v {
TierTargets::List(l) => l.clone(),
TierTargets::One(s) => vec![s.clone()],
};
(k.clone(), list)
})
.collect()
}
fn parse_routing(model: &str) -> RoutingDirective {
if let Some(rest) = model.strip_prefix("cortiq-auto") {
let profile = rest.strip_prefix(':').map(|p| p.to_string());
RoutingDirective::Auto { profile }
} else {
RoutingDirective::Pinned {
model_id: model.to_string(),
}
}
}
// ───────────────────────────── router ───────────────────────────────────────
/// Admin API routes (token-protected). Mounted into the main router.
/// The token is captured in a middleware closure so it does not depend on State.
pub fn api_routes(admin_token: String) -> Router<SharedState> {
Router::new()
.route("/admin/api/health", get(health))
.route("/admin/api/meta", get(meta))
.route("/admin/api/config", get(get_config).put(put_config))
.route("/admin/api/models", get(list_models).post(create_model))
.route(
"/admin/api/models/:id",
axum::routing::put(update_model).delete(delete_model),
)
.route("/admin/api/models/:id/probe", post(probe_model))
.route("/admin/api/routing", get(get_routing).put(put_routing))
.route(
"/admin/api/protocols",
get(get_protocols).put(put_protocols),
)
.route("/admin/api/settings", get(get_settings).put(put_settings))
.route("/admin/api/keys", get(list_keys).post(create_key))
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
.route(
"/admin/api/secrets",
get(list_secrets).put(set_secret).delete(clear_secret),
)
.route("/admin/api/stats", get(get_stats))
.route("/admin/api/requests", get(get_requests))
.route("/admin/api/test", post(run_test))
.route_layer(middleware::from_fn(move |req: Request, next: Next| {
let expected = admin_token.clone();
async move { auth(expected, req, next).await }
}))
}
/// Extract Bearer admin token from the `Authorization` header.
fn bearer_token(req: &Request) -> String {
req.headers()
.get(header::AUTHORIZATION)
.and_then(|v| v.to_str().ok())
.map(|v| {
v.strip_prefix("Bearer ")
.or_else(|| v.strip_prefix("bearer "))
.unwrap_or(v)
.trim()
})
.unwrap_or("")
.to_string()
}
/// Admin authentication middleware (constant-time comparison).
async fn auth(expected: String, req: Request, next: Next) -> Response {
let token = bearer_token(&req);
if !expected.is_empty() && constant_time_eq(token.as_bytes(), expected.as_bytes()) {
next.run(req).await
} else {
(
StatusCode::UNAUTHORIZED,
Json(json!({"error": {"message": "invalid or missing admin token", "type": "authentication_error", "code": "unauthorized"}})),
)
.into_response()
}
}
// ───────────────────────────── metrics ──────────────────────────────────────
/// `GET /metrics` (Prometheus). Exposed when `telemetry.metrics = true`.
pub async fn metrics(State(state): State<SharedState>) -> Response {
let live = state.live();
if !live.cfg.telemetry.metrics {
return (StatusCode::NOT_FOUND, "metrics disabled").into_response();
}
let body = state.stats.prometheus();
([(header::CONTENT_TYPE, "text/plain; version=0.0.4")], body).into_response()
}
// ───────────────────────────── health / meta ───────────────────────────────
async fn probe_router(url: &str) -> (bool, u64) {
let client = match reqwest::Client::builder()
.timeout(std::time::Duration::from_millis(1500))
.danger_accept_invalid_certs(true)
.build()
{
Ok(c) => c,
Err(_) => return (false, 0),
};
let started = Instant::now();
let target = format!("{}/healthz", url.trim_end_matches('/'));
let ok = client
.get(&target)
.send()
.await
.map(|r| r.status().as_u16() < 500)
.unwrap_or(false);
(ok, started.elapsed().as_millis() as u64)
}
async fn health(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let (router_ok, router_ms) = probe_router(&live.cfg.router.url).await;
let models: Vec<Value> = live
.cfg
.models
.iter()
.map(|m| {
let key = match &m.api_key_env {
Some(env) => state.secrets.source(env),
None => "none",
};
json!({
"id": m.id,
"provider": m.provider,
"model": m.model,
"kind": m.kind,
"cost_tier": m.cost_tier,
"key_source": key,
"in_registry": live.registry.get(&m.id).is_some(),
})
})
.collect();
ok(json!({
"ok": true,
"version": env!("CARGO_PKG_VERSION"),
"listen": live.cfg.listen,
"router": { "url": live.cfg.router.url, "reachable": router_ok, "latency_ms": router_ms },
"models": models,
}))
}
async fn meta() -> ApiResult {
ok(json!({
"version": env!("CARGO_PKG_VERSION"),
"providers": ["openai", "anthropic", "ollama", "http"],
"provider_impl": { "openai": true, "ollama": true, "http": true, "anthropic": false },
"cost_tiers": ["cheap", "mid", "expensive"],
"kinds": ["chat", "embedding"],
"profiles": ["cost-saver", "balanced", "quality-first"],
"text_strategies": ["last_user", "last_user_plus_system", "concat_all"],
"policy_modes": ["fixed_table", "cost_aware"],
"tiers": ["low", "medium", "high"],
"caps": ["tools", "vision"],
"protocols_impl": {
"openai_chat": true, "openai_completions": false, "openai_embeddings": false,
"openai_models": false, "anthropic_messages": false, "mcp": false, "native_passthrough": false
},
"languages": ["en", "ru", "de", "fr", "es", "zh", "tr"],
}))
}
// ───────────────────────────── config ──────────────────────────────────────
async fn get_config(State(state): State<SharedState>) -> ApiResult {
let cfg = current_cfg(&state);
ok(serde_json::to_value(&cfg).map_err(|e| ApiError::bad(e.to_string()))?)
}
async fn put_config(State(state): State<SharedState>, Json(cfg): Json<Config>) -> ApiResult {
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
// ───────────────────────────── models ──────────────────────────────────────
async fn list_models(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let models: Vec<Value> = live
.cfg
.models
.iter()
.map(|m| {
let mut v = serde_json::to_value(m).unwrap_or(json!({}));
let key_source = match &m.api_key_env {
Some(env) => state.secrets.source(env),
None => "none",
};
v["key_source"] = json!(key_source);
v
})
.collect();
ok(json!({ "models": models }))
}
async fn create_model(State(state): State<SharedState>, Json(m): Json<ModelCfg>) -> ApiResult {
let mut cfg = current_cfg(&state);
if cfg.models.iter().any(|x| x.id == m.id) {
return Err(ApiError::bad(format!("model id '{}' already exists", m.id)));
}
cfg.models.push(m);
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
async fn update_model(
State(state): State<SharedState>,
Path(id): Path<String>,
Json(m): Json<ModelCfg>,
) -> ApiResult {
let mut cfg = current_cfg(&state);
let idx = cfg
.models
.iter()
.position(|x| x.id == id)
.ok_or_else(|| ApiError::not_found(format!("model '{id}' not found")))?;
// if the id was renamed — must not conflict with an existing one
if m.id != id && cfg.models.iter().any(|x| x.id == m.id) {
return Err(ApiError::bad(format!("model id '{}' already exists", m.id)));
}
cfg.models[idx] = m;
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
async fn delete_model(State(state): State<SharedState>, Path(id): Path<String>) -> ApiResult {
let mut cfg = current_cfg(&state);
let before = cfg.models.len();
cfg.models.retain(|x| x.id != id);
if cfg.models.len() == before {
return Err(ApiError::not_found(format!("model '{id}' not found")));
}
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
async fn probe_model(State(state): State<SharedState>, Path(id): Path<String>) -> ApiResult {
let live = state.live();
let mcfg = live
.cfg
.models
.iter()
.find(|m| m.id == id)
.ok_or_else(|| ApiError::not_found(format!("model '{id}' not found")))?
.clone();
if mcfg.provider == "anthropic" {
return ok(json!({
"ok": false,
"error": "anthropic provider not yet implemented (planned v0.2)",
}));
}
let provider = live
.registry
.get(&id)
.ok_or_else(|| ApiError::bad(format!("model '{id}' has no runtime provider")))?;
let req = ChatRequest {
routing: RoutingDirective::Pinned {
model_id: id.clone(),
},
messages: vec![Message {
role: "user".into(),
content: "ping".into(),
tool_calls: vec![],
}],
tools: vec![],
params: GenParams {
temperature: Some(0.0),
max_tokens: Some(1),
..Default::default()
},
stream: false,
meta: RequestMeta {
protocol: "probe".into(),
..Default::default()
},
};
let started = Instant::now();
let res = provider.chat(req).await;
let latency_ms = started.elapsed().as_millis() as u64;
match res {
Ok(_) => ok(json!({ "ok": true, "latency_ms": latency_ms })),
Err(e) => ok(json!({ "ok": false, "latency_ms": latency_ms, "error": e.to_string() })),
}
}
// ───────────────────────────── routing ─────────────────────────────────────
async fn get_routing(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
ok(json!({
"tiers": tiers_to_map(&live.cfg.routing),
"default": live.cfg.routing.default,
"policy": live.cfg.routing.policy,
}))
}
#[derive(Deserialize)]
struct RoutingBody {
#[serde(default)]
tiers: HashMap<String, Vec<String>>,
default: String,
#[serde(default)]
policy: RoutingPolicy,
}
async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingBody>) -> ApiResult {
let mut cfg = current_cfg(&state);
cfg.routing = RoutingCfg {
tiers: body
.tiers
.into_iter()
.map(|(k, v)| (k, TierTargets::List(v)))
.collect(),
default: body.default,
policy: body.policy,
};
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
// ───────────────────────────── protocols ───────────────────────────────────
async fn get_protocols(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
ok(serde_json::to_value(&live.cfg.protocols).unwrap_or(json!({})))
}
async fn put_protocols(State(state): State<SharedState>, Json(p): Json<ProtocolsCfg>) -> ApiResult {
let mut cfg = current_cfg(&state);
cfg.protocols = p;
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
// ───────────────────────────── settings ────────────────────────────────────
async fn get_settings(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let c = &live.cfg;
// admin section without token value (not stored here — only the env name)
ok(json!({
"listen": c.listen,
"router": c.router,
"route": c.route,
"breaker": c.breaker,
"log": c.log,
"telemetry": c.telemetry,
"cortiq": c.cortiq,
"stats": c.stats,
"admin": c.admin,
}))
}
#[derive(Deserialize)]
struct SettingsBody {
listen: Option<String>,
router: Option<RouterCfg>,
route: Option<RouteCfg>,
breaker: Option<BreakerCfg>,
log: Option<LogCfg>,
telemetry: Option<TelemetryCfg>,
cortiq: Option<CortiqCfg>,
stats: Option<StatsCfg>,
admin: Option<AdminCfg>,
}
async fn put_settings(State(state): State<SharedState>, Json(b): Json<SettingsBody>) -> ApiResult {
let mut cfg = current_cfg(&state);
let mut needs_restart = false;
if let Some(v) = b.listen {
if v != cfg.listen {
needs_restart = true;
}
cfg.listen = v;
}
if let Some(v) = b.router {
cfg.router = v;
}
if let Some(v) = b.route {
cfg.route = v;
}
if let Some(v) = b.breaker {
cfg.breaker = v;
}
if let Some(v) = b.log {
cfg.log = v;
}
if let Some(v) = b.telemetry {
cfg.telemetry = v;
}
if let Some(v) = b.cortiq {
cfg.cortiq = v;
}
if let Some(v) = b.stats {
cfg.stats = v;
}
if let Some(v) = b.admin {
if v.listen != cfg.admin.listen {
needs_restart = true;
}
cfg.admin = v;
}
state.reload(cfg)?;
ok(json!({ "ok": true, "needs_restart": needs_restart }))
}
// ───────────────────────────── keys ────────────────────────────────────────
async fn list_keys(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let keys: Vec<Value> = live
.cfg
.api_keys
.iter()
.map(|k| {
json!({
"key": k.key,
"key_masked": mask_key(&k.key),
"account": k.account,
"rate_per_min": k.rate_per_min,
"allow_models": k.allow_models,
})
})
.collect();
ok(json!({ "keys": keys, "open_mode": live.cfg.api_keys.is_empty() }))
}
#[derive(Deserialize)]
struct NewKey {
#[serde(default)]
key: String,
account: String,
#[serde(default)]
rate_per_min: u32,
#[serde(default)]
allow_models: Vec<String>,
}
async fn create_key(State(state): State<SharedState>, Json(b): Json<NewKey>) -> ApiResult {
let key = if b.key.trim().is_empty() {
format!("sk-gw-{}", random_token(18))
} else {
b.key.trim().to_string()
};
let mut cfg = current_cfg(&state);
if cfg.api_keys.iter().any(|k| k.key == key) {
return Err(ApiError::bad("key already exists"));
}
cfg.api_keys.push(ApiKeyCfg {
key: key.clone(),
account: b.account,
rate_per_min: b.rate_per_min,
allow_models: b.allow_models,
});
state.reload(cfg)?;
// show the full key only once
ok(json!({ "ok": true, "key": key }))
}
async fn delete_key(State(state): State<SharedState>, Path(key): Path<String>) -> ApiResult {
let mut cfg = current_cfg(&state);
let before = cfg.api_keys.len();
cfg.api_keys.retain(|k| k.key != key);
if cfg.api_keys.len() == before {
return Err(ApiError::not_found("key not found"));
}
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
// ───────────────────────────── secrets ─────────────────────────────────────
async fn list_secrets(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
// collect all env names referenced in the config
let mut names: Vec<String> = Vec::new();
for m in &live.cfg.models {
if let Some(env) = &m.api_key_env {
if !names.contains(env) {
names.push(env.clone());
}
}
}
if let Some(env) = &live.cfg.router.api_key_env {
if !names.contains(env) {
names.push(env.clone());
}
}
let secrets: Vec<Value> = names
.iter()
.map(|n| json!({ "name": n, "source": state.secrets.source(n) }))
.collect();
ok(json!({ "secrets": secrets }))
}
#[derive(Deserialize)]
struct SecretBody {
name: String,
value: String,
}
async fn set_secret(State(state): State<SharedState>, Json(b): Json<SecretBody>) -> ApiResult {
if b.name.trim().is_empty() {
return Err(ApiError::bad("secret name is required"));
}
state.secrets.set(b.name.trim(), &b.value)?;
// rebuild providers with the new key (without writing TOML)
state.rebuild()?;
ok(json!({ "ok": true }))
}
#[derive(Deserialize)]
struct NameQuery {
name: String,
}
async fn clear_secret(State(state): State<SharedState>, Query(q): Query<NameQuery>) -> ApiResult {
state.secrets.clear(q.name.trim())?;
state.rebuild()?;
ok(json!({ "ok": true }))
}
// ───────────────────────────── stats / requests ────────────────────────────
#[derive(Deserialize)]
struct StatsQuery {
#[serde(default)]
range: Option<String>,
#[serde(default)]
groupby: Option<String>,
}
async fn get_stats(State(state): State<SharedState>, Query(q): Query<StatsQuery>) -> ApiResult {
let range_secs = q
.range
.as_deref()
.and_then(parse_duration_secs)
.unwrap_or(24 * 3600);
let groupby = q.groupby.as_deref().unwrap_or("model");
ok(state.stats.snapshot(range_secs, groupby))
}
#[derive(Deserialize)]
struct RecentQuery {
#[serde(default)]
limit: Option<usize>,
#[serde(default)]
offset: Option<usize>,
}
async fn get_requests(State(state): State<SharedState>, Query(q): Query<RecentQuery>) -> ApiResult {
let limit = q.limit.unwrap_or(50).min(500);
let offset = q.offset.unwrap_or(0);
let items = state.stats.recent(limit, offset);
ok(json!({ "requests": items }))
}
// ───────────────────────────── playground (test) ───────────────────────────
#[derive(Deserialize)]
struct TestBody {
#[serde(default = "default_test_model")]
model: String,
messages: Vec<Message>,
#[serde(default)]
temperature: Option<f32>,
#[serde(default)]
max_tokens: Option<u32>,
}
fn default_test_model() -> String {
"cortiq-auto".into()
}
async fn run_test(State(state): State<SharedState>, Json(b): Json<TestBody>) -> ApiResult {
if b.messages.is_empty() {
return Err(ApiError::bad("messages must not be empty"));
}
let req = ChatRequest {
routing: parse_routing(&b.model),
messages: b.messages,
tools: vec![],
params: GenParams {
temperature: b.temperature,
max_tokens: b.max_tokens,
..Default::default()
},
stream: false,
meta: RequestMeta {
account: "playground".into(),
protocol: "playground".into(),
..Default::default()
},
};
let started = Instant::now();
let res = state.pipeline.run(req, &state).await;
let latency_ms = started.elapsed().as_millis() as u64;
match res {
Ok(resp) => {
let answer = resp
.choices
.first()
.map(|c| c.message.content.clone())
.unwrap_or_default();
ok(json!({
"ok": true,
"latency_ms": latency_ms,
"answer": answer,
"model_used": resp.model_used,
"usage": {
"prompt_tokens": resp.usage.prompt_tokens,
"completion_tokens": resp.usage.completion_tokens,
"total_tokens": resp.usage.total_tokens,
},
"cortiq": {
"task_label": resp.cortiq.task_label,
"complexity": { "score": resp.cortiq.complexity_score, "tier": resp.cortiq.complexity_tier },
"selected_model": resp.cortiq.selected_model,
"route_source": resp.cortiq.route_source,
"router_request_id": resp.cortiq.router_request_id,
"cost_usd": resp.cortiq.cost_usd,
"failover": resp.cortiq.failover,
},
}))
}
Err(e) => ok(json!({ "ok": false, "latency_ms": latency_ms, "error": e.to_string() })),
}
}
src/config.rs
+474
View File
@@ -0,0 +1,474 @@
//! Configuration loading from TOML. Structs mirror config/gateway.example.toml.
//! Secrets are read from environment variables by the names given in `*_env` fields.
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Config {
pub listen: String,
pub router: RouterCfg,
#[serde(default)]
pub route: RouteCfg,
#[serde(default)]
pub models: Vec<ModelCfg>,
pub routing: RoutingCfg,
#[serde(default)]
pub breaker: BreakerCfg,
#[serde(default)]
pub protocols: ProtocolsCfg,
#[serde(default)]
pub cortiq: CortiqCfg,
#[serde(default)]
pub api_keys: Vec<ApiKeyCfg>,
#[serde(default)]
pub idempotency: IdempotencyCfg,
#[serde(default)]
pub log: LogCfg,
#[serde(default)]
pub telemetry: TelemetryCfg,
#[serde(default)]
pub admin: AdminCfg,
#[serde(default)]
pub stats: StatsCfg,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RouterCfg {
pub url: String,
#[serde(default = "default_router_timeout")]
pub timeout_ms: u64,
#[serde(default = "default_verify_tls")]
pub verify_tls: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub api_key_env: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub taxonomy_id: Option<String>,
}
fn default_router_timeout() -> u64 {
800
}
fn default_verify_tls() -> bool {
true
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RouteCfg {
#[serde(default = "default_text_strategy")]
pub text_strategy: String,
#[serde(default = "default_max_chars")]
pub max_chars: usize,
#[serde(default = "default_cache_ttl")]
pub cache_ttl: String,
#[serde(default = "default_profile")]
pub profile: String,
}
impl Default for RouteCfg {
fn default() -> Self {
Self {
text_strategy: default_text_strategy(),
max_chars: default_max_chars(),
cache_ttl: default_cache_ttl(),
profile: default_profile(),
}
}
}
fn default_text_strategy() -> String {
"last_user".into()
}
fn default_max_chars() -> usize {
4000
}
fn default_cache_ttl() -> String {
"60s".into()
}
fn default_profile() -> String {
"balanced".into()
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct ModelCfg {
pub id: String,
pub provider: String, // openai | anthropic | ollama | http
pub base_url: String,
pub model: String,
#[serde(default)]
pub cost_tier: String,
#[serde(default)]
pub price_in: f64,
#[serde(default)]
pub price_out: f64,
#[serde(default = "default_kind")]
pub kind: String, // chat | embedding
#[serde(default, skip_serializing_if = "Option::is_none")]
pub api_key_env: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub caps: Vec<String>,
}
fn default_kind() -> String {
"chat".into()
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RoutingCfg {
#[serde(flatten)]
pub tiers: HashMap<String, TierTargets>,
pub default: String,
#[serde(default)]
pub policy: RoutingPolicy,
}
/// A routing tier is either a list of model_ids or the string `default`.
/// (`default` is handled by a separate field; only tier lists appear here.)
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(untagged)]
pub enum TierTargets {
List(Vec<String>),
One(String),
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RoutingPolicy {
#[serde(default = "default_mode")]
pub mode: String, // fixed_table | cost_aware
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_cost_usd_per_request: Option<f64>,
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
pub min_class: HashMap<String, String>,
}
impl Default for RoutingPolicy {
fn default() -> Self {
Self {
mode: default_mode(),
min_class: HashMap::new(),
max_cost_usd_per_request: None,
}
}
}
fn default_mode() -> String {
"fixed_table".into()
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct BreakerCfg {
#[serde(default = "default_breaker_threshold")]
pub threshold: u32,
#[serde(default = "default_breaker_cooldown")]
pub cooldown: String,
}
impl Default for BreakerCfg {
fn default() -> Self {
Self {
threshold: default_breaker_threshold(),
cooldown: default_breaker_cooldown(),
}
}
}
fn default_breaker_threshold() -> u32 {
5
}
fn default_breaker_cooldown() -> String {
"30s".into()
}
#[derive(Clone, Debug, Serialize, Deserialize, Default)]
pub struct ProtocolsCfg {
#[serde(default)]
pub openai_chat: bool,
#[serde(default)]
pub openai_completions: bool,
#[serde(default)]
pub openai_embeddings: bool,
#[serde(default)]
pub openai_models: bool,
#[serde(default)]
pub anthropic_messages: bool,
#[serde(default)]
pub mcp: bool,
#[serde(default)]
pub native_passthrough: bool,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct CortiqCfg {
#[serde(default)]
pub echo: bool,
}
impl Default for CortiqCfg {
fn default() -> Self {
Self { echo: true }
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct ApiKeyCfg {
pub key: String,
pub account: String,
#[serde(default)]
pub rate_per_min: u32,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub allow_models: Vec<String>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct IdempotencyCfg {
#[serde(default = "default_idem_ttl")]
pub ttl: String,
}
impl Default for IdempotencyCfg {
fn default() -> Self {
Self {
ttl: default_idem_ttl(),
}
}
}
fn default_idem_ttl() -> String {
"10m".into()
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct LogCfg {
#[serde(default)]
pub bodies: bool,
#[serde(default = "default_log_level")]
pub level: String,
}
impl Default for LogCfg {
fn default() -> Self {
Self {
bodies: false,
level: default_log_level(),
}
}
}
fn default_log_level() -> String {
"info".into()
}
#[derive(Clone, Debug, Serialize, Deserialize, Default)]
pub struct TelemetryCfg {
#[serde(default)]
pub metrics: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub otlp_endpoint_env: Option<String>,
}
/// Access to the web management panel and admin API.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct AdminCfg {
/// Whether the admin panel and admin API (`/admin`, `/admin/api/*`) are enabled.
#[serde(default = "default_admin_enabled")]
pub enabled: bool,
/// Name of the env variable holding the admin token. If unset (or empty),
/// a token is generated at startup and printed to the log.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub token_env: Option<String>,
/// Optional separate bind address for the admin surface (e.g. `127.0.0.1:9001`).
/// If unset, the admin interface lives on the main `listen` address under `/admin`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub listen: Option<String>,
}
impl Default for AdminCfg {
fn default() -> Self {
Self {
enabled: default_admin_enabled(),
token_env: None,
listen: None,
}
}
}
fn default_admin_enabled() -> bool {
true
}
/// Request statistics tracking (for the dashboard and `/metrics`).
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct StatsCfg {
/// Whether to track request statistics.
#[serde(default = "default_stats_enabled")]
pub enabled: bool,
/// Append-only request log file (JSONL). Empty means in-memory only.
#[serde(default = "default_stats_file")]
pub file: String,
/// How many recent requests to keep in memory (the "recent" ring buffer).
#[serde(default = "default_stats_ring")]
pub ring_size: usize,
/// Retention window for aggregates/time-series replayed on startup, e.g. `7d`.
#[serde(default = "default_stats_retention")]
pub retention: String,
}
impl Default for StatsCfg {
fn default() -> Self {
Self {
enabled: default_stats_enabled(),
file: default_stats_file(),
ring_size: default_stats_ring(),
retention: default_stats_retention(),
}
}
}
fn default_stats_enabled() -> bool {
true
}
fn default_stats_file() -> String {
"config/stats.jsonl".into()
}
fn default_stats_ring() -> usize {
500
}
fn default_stats_retention() -> String {
"7d".into()
}
impl Config {
/// Load and validate the config from a file.
pub fn load(path: &str) -> anyhow::Result<Self> {
let raw = std::fs::read_to_string(path)
.map_err(|e| anyhow::anyhow!("cannot read config {path}: {e}"))?;
let cfg: Config = toml::from_str(&raw)?;
cfg.validate()?;
Ok(cfg)
}
/// Integrity validation: non-empty model pool, valid routing targets.
pub fn validate(&self) -> anyhow::Result<()> {
if self.models.is_empty() {
anyhow::bail!("config: at least one [[models]] entry is required");
}
// uniqueness of model ids
let mut seen = std::collections::HashSet::new();
for m in &self.models {
if !seen.insert(m.id.as_str()) {
anyhow::bail!("config: duplicate model id '{}'", m.id);
}
}
// every target in [routing] must exist in the model pool
let ids: std::collections::HashSet<&str> =
self.models.iter().map(|m| m.id.as_str()).collect();
if !ids.contains(self.routing.default.as_str()) {
anyhow::bail!(
"config: routing.default '{}' is not a known model id",
self.routing.default
);
}
for (tier, targets) in &self.routing.tiers {
let list = match targets {
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
};
for id in &list {
if !ids.contains(id.as_str()) {
anyhow::bail!(
"config: routing.{} references unknown model id '{}'",
tier,
id
);
}
}
}
Ok(())
}
/// Serialize the config to a TOML string (no secrets — they are intentionally absent here).
pub fn to_toml_string(&self) -> anyhow::Result<String> {
let header = "# Cortiq Gateway — configuration (managed via the admin panel).\n\
# This file is rewritten by the panel; manual edits are preserved\n\
# only until the next change made through the UI.\n\n";
let body = toml::to_string_pretty(self)?;
Ok(format!("{header}{body}"))
}
/// Atomically save the config to a file (write to a temporary file then rename).
pub fn save(&self, path: &str) -> anyhow::Result<()> {
self.validate()?;
let data = self.to_toml_string()?;
let p = std::path::Path::new(path);
let dir = p.parent().unwrap_or_else(|| std::path::Path::new("."));
std::fs::create_dir_all(dir).ok();
let tmp = p.with_extension("toml.tmp");
std::fs::write(&tmp, data.as_bytes())
.map_err(|e| anyhow::anyhow!("cannot write config tmp {}: {e}", tmp.display()))?;
std::fs::rename(&tmp, p)
.map_err(|e| anyhow::anyhow!("cannot replace config {path}: {e}"))?;
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
const SAMPLE: &str = r#"
listen = "0.0.0.0:9000"
[router]
url = "http://localhost:8080"
api_key_env = "CORTIQ_ROUTER_KEY"
[[models]]
id = "local-qwen"
provider = "openai"
base_url = "http://localhost:8000/v1"
model = "qwen2.5-7b-instruct"
cost_tier = "cheap"
caps = ["tools"]
[[models]]
id = "gpt-4o-mini"
provider = "openai"
base_url = "https://api.openai.com/v1"
model = "gpt-4o-mini"
api_key_env = "OPENAI_API_KEY"
cost_tier = "mid"
price_in = 0.15
price_out = 0.6
[routing]
low = ["local-qwen"]
medium = ["gpt-4o-mini", "local-qwen"]
default = "gpt-4o-mini"
[routing.policy]
mode = "cost_aware"
max_cost_usd_per_request = 0.5
min_class = { low = "cheap", medium = "mid" }
"#;
#[test]
fn roundtrip_preserves_routing_and_models() {
let cfg: Config = toml::from_str(SAMPLE).expect("parse sample");
cfg.validate().expect("sample valid");
// key risk: serialization of flattened [routing] + nested policy
let serialized = cfg.to_toml_string().expect("serialize");
let back: Config = toml::from_str(&serialized).expect("re-parse serialized");
back.validate().expect("roundtrip valid");
assert_eq!(back.listen, "0.0.0.0:9000");
assert_eq!(back.models.len(), 2);
assert_eq!(back.routing.default, "gpt-4o-mini");
assert_eq!(back.routing.policy.mode, "cost_aware");
assert_eq!(back.routing.policy.max_cost_usd_per_request, Some(0.5));
assert_eq!(
back.routing.policy.min_class.get("low").map(String::as_str),
Some("cheap")
);
let low = back.routing.tiers.get("low").expect("low tier present");
match low {
TierTargets::List(v) => assert_eq!(v, &vec!["local-qwen".to_string()]),
TierTargets::One(s) => assert_eq!(s, "local-qwen"),
}
// admin/stats sections receive defaults
assert!(back.admin.enabled);
assert!(back.stats.enabled);
}
#[test]
fn validate_rejects_unknown_routing_target() {
let mut cfg: Config = toml::from_str(SAMPLE).unwrap();
cfg.routing.default = "does-not-exist".into();
assert!(cfg.validate().is_err());
}
}
src/error.rs
+74
View File
@@ -0,0 +1,74 @@
//! Unified gateway error type + rendering into an OpenAI-compatible envelope
//! `{"error": {message, type, code, param}}` that SDK clients parse natively.
//! See docs/PROTOCOLS.md §8.5.
use axum::{http::StatusCode, response::IntoResponse, Json};
use serde_json::json;
#[derive(Debug, thiserror::Error)]
pub enum GatewayError {
#[error("{0}")]
InvalidRequest(String),
#[error("authentication failed: {0}")]
Unauthorized(String),
#[error("rate limit exceeded")]
RateLimited { retry_after_secs: u64 },
#[error("upstream unavailable: {0}")]
UpstreamUnavailable(String),
#[error("internal error: {0}")]
Internal(String),
}
impl GatewayError {
fn parts(&self) -> (StatusCode, &'static str, &'static str) {
match self {
Self::InvalidRequest(_) => (
StatusCode::BAD_REQUEST,
"invalid_request_error",
"invalid_request",
),
Self::Unauthorized(_) => (
StatusCode::UNAUTHORIZED,
"authentication_error",
"unauthorized",
),
Self::RateLimited { .. } => (
StatusCode::TOO_MANY_REQUESTS,
"rate_limit_error",
"rate_limited",
),
Self::UpstreamUnavailable(_) => (
StatusCode::BAD_GATEWAY,
"upstream_unavailable",
"no_healthy_model",
),
Self::Internal(_) => (
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
"internal",
),
}
}
}
impl IntoResponse for GatewayError {
fn into_response(self) -> axum::response::Response {
let (status, typ, code) = self.parts();
let body = Json(json!({
"error": {
"message": self.to_string(),
"type": typ,
"code": code,
"param": serde_json::Value::Null,
}
}));
let mut resp = (status, body).into_response();
if let Self::RateLimited { retry_after_secs } = self {
resp.headers_mut()
.insert("retry-after", retry_after_secs.to_string().parse().unwrap());
}
resp
}
}
pub type Result<T> = std::result::Result<T, GatewayError>;
src/main.rs
+107
View File
@@ -0,0 +1,107 @@
//! Cortiq Gateway — entry point.
//!
//! Loads the config, builds the shared state, assembles routes from protocols and
//! the admin panel, and starts the HTTP server. See docs/ARCHITECTURE.md.
// Part of the canonical model / error API surface (auth, rate-limit, idempotency,
// traceparent, vision/streaming caps) is intentionally reserved for upcoming phases;
// allow the unused items until those features land.
#![allow(dead_code)]
mod admin;
mod config;
mod error;
mod model;
mod pipeline;
mod protocols;
mod providers;
mod registry;
mod router_client;
mod routing;
mod secrets;
mod state;
mod stats;
use axum::{routing::get, Router};
use clap::Parser;
use config::Config;
use state::SharedState;
#[derive(Parser)]
#[command(
name = "cortiq-gateway",
version,
about = "Universal LLM gateway with intelligent routing"
)]
struct Args {
/// Path to the TOML config file.
#[arg(long, default_value = "config/gateway.toml")]
config: String,
/// Admin token for the web panel (overrides `[admin].token_env`).
#[arg(long)]
admin_token: Option<String>,
}
/// Resolve the admin token: env from `[admin].token_env` → `--admin-token` →
/// generate a random one. Returns `(token, generated)`.
fn resolve_admin_token(cfg: &Config, args: &Args) -> (String, bool) {
if let Some(env) = &cfg.admin.token_env {
if let Ok(v) = std::env::var(env) {
if !v.is_empty() {
return (v, false);
}
}
}
if let Some(t) = &args.admin_token {
if !t.is_empty() {
return (t.clone(), false);
}
}
(admin::random_token(24), true)
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let args = Args::parse();
let cfg = Config::load(&args.config)?;
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| cfg.log.level.clone().into()),
)
.init();
let listen = cfg.listen.clone();
let admin_enabled = cfg.admin.enabled;
let (admin_token, generated) = resolve_admin_token(&cfg, &args);
let state = SharedState::build(cfg, args.config.clone())?;
let mut app = Router::new()
.route("/healthz", get(|| async { "ok" }))
.route("/readyz", get(|| async { "ready" }))
.route("/metrics", get(admin::metrics))
.merge(protocols::build_router());
if admin_enabled {
app = app
.merge(admin::api_routes(admin_token.clone()))
.fallback(admin::assets::fallback);
}
let app = app.with_state(state);
let listener = tokio::net::TcpListener::bind(&listen).await?;
tracing::info!("cortiq-gateway listening on {listen}");
if admin_enabled {
tracing::info!("admin console: http://{listen}/admin");
if generated {
tracing::warn!(
"admin token (auto-generated, set [admin].token_env to persist): {admin_token}"
);
}
}
axum::serve(listener, app).await?;
Ok(())
}
src/model.rs
+102
View File
@@ -0,0 +1,102 @@
//! Canonical (protocol-neutral) request/response model.
//!
//! All incoming protocols are translated into these types, and providers translate
//! from them. This way N protocols × M providers do not become N×M translations:
//! each adapter only knows its own protocol ↔ canonical mapping. See docs/ARCHITECTURE.md §3.
use serde::{Deserialize, Serialize};
/// How to handle routing for a specific request.
#[derive(Clone, Debug)]
pub enum RoutingDirective {
/// `model = "cortiq-auto"` — ask cortiq-router and pick a model from the pool.
Auto { profile: Option<String> },
/// `model = "<real id>"` — call a specific model directly, bypassing routing.
Pinned { model_id: String },
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Message {
pub role: String, // system | user | assistant | tool
#[serde(default)]
pub content: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub tool_calls: Vec<serde_json::Value>,
}
#[derive(Clone, Debug, Default)]
pub struct GenParams {
pub temperature: Option<f32>,
pub max_tokens: Option<u32>,
pub top_p: Option<f32>,
pub stop: Vec<String>,
/// Fields we do not interpret but must proxy through to the provider.
pub passthrough: serde_json::Map<String, serde_json::Value>,
}
#[derive(Clone, Debug, Default)]
pub struct RequestMeta {
pub account: String,
pub protocol: String,
pub idempotency_key: Option<String>,
pub traceparent: Option<String>,
}
/// Canonical form of a generation request.
#[derive(Clone, Debug)]
pub struct ChatRequest {
pub routing: RoutingDirective,
pub messages: Vec<Message>,
pub tools: Vec<serde_json::Value>,
pub params: GenParams,
pub stream: bool,
pub meta: RequestMeta,
}
#[derive(Clone, Debug, Serialize)]
pub struct Usage {
pub prompt_tokens: u32,
pub completion_tokens: u32,
pub total_tokens: u32,
}
/// Routing metadata attached to the response (X-Cortiq-* headers and the `cortiq` field).
#[derive(Clone, Debug, Serialize)]
pub struct RouteInfo {
pub task_label: String,
pub complexity_score: f32,
pub complexity_tier: String,
pub selected_model: String,
pub route_source: String, // router | cache | fallback | pinned
pub router_request_id: Option<String>,
pub cost_usd: f64,
#[serde(default)]
pub failover: bool,
}
#[derive(Clone, Debug, Serialize)]
pub struct Choice {
pub index: u32,
pub message: Message,
pub finish_reason: String,
}
/// Canonical form of a response.
#[derive(Clone, Debug)]
pub struct ChatResponse {
pub id: String,
pub model_used: String,
pub choices: Vec<Choice>,
pub usage: Usage,
pub cortiq: RouteInfo,
}
/// Decision from cortiq-router (the subset of its contract needed by the gateway).
#[derive(Clone, Debug)]
pub struct RouteDecision {
pub task_label: String,
pub complexity_score: f32,
pub complexity_tier: String,
pub router_request_id: Option<String>,
pub source: String, // router | cache | fallback
}
src/pipeline.rs
+251
View File
@@ -0,0 +1,251 @@
//! Canonical request processing pipeline (Pipeline).
//! Coordinates stages: Extract -> Route -> Select -> Translate & Call,
//! and also measures latency and records per-request statistics.
use crate::error::{GatewayError, Result};
use crate::model::{
ChatRequest, ChatResponse, Message, RouteDecision, RouteInfo, RoutingDirective,
};
use crate::state::SharedState;
use crate::stats::RequestRecord;
use std::time::{Instant, SystemTime, UNIX_EPOCH};
pub struct Pipeline;
impl Pipeline {
pub fn new() -> Self {
Self
}
/// Run the routing and generation pipeline.
pub async fn run(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
let started = Instant::now();
let account = req.meta.account.clone();
let protocol = req.meta.protocol.clone();
let directive = match &req.routing {
RoutingDirective::Pinned { .. } => "pinned",
RoutingDirective::Auto { .. } => "auto",
}
.to_string();
let result = self.run_inner(req, state).await;
// record statistics (for both success and error)
let latency_ms = started.elapsed().as_millis() as u64;
let ts = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs();
let rec = match &result {
Ok(resp) => RequestRecord {
ts,
account,
protocol,
directive,
task_label: resp.cortiq.task_label.clone(),
tier: resp.cortiq.complexity_tier.clone(),
score: resp.cortiq.complexity_score,
model_id: resp.cortiq.selected_model.clone(),
route_source: resp.cortiq.route_source.clone(),
prompt_tokens: resp.usage.prompt_tokens,
completion_tokens: resp.usage.completion_tokens,
cost_usd: resp.cortiq.cost_usd,
latency_ms,
outcome: "ok".to_string(),
failover: resp.cortiq.failover,
error: None,
},
Err(err) => RequestRecord {
ts,
account,
protocol,
directive,
task_label: String::new(),
tier: String::new(),
score: 0.0,
model_id: String::new(),
route_source: String::new(),
prompt_tokens: 0,
completion_tokens: 0,
cost_usd: 0.0,
latency_ms,
outcome: "error".to_string(),
failover: false,
error: Some(err.to_string()),
},
};
state.stats.record(rec);
result
}
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
let live = state.live();
// 1. Extract text for classification
let text = self.extract_text(
&req.messages,
&live.cfg.route.text_strategy,
live.cfg.route.max_chars,
);
// 2. Obtain the router decision (or use the pinned model)
let (decision, route_source) = match &req.routing {
RoutingDirective::Pinned { model_id: _ } => {
let dec = RouteDecision {
task_label: "pinned".to_string(),
complexity_score: 0.0,
complexity_tier: "pinned".to_string(),
router_request_id: None,
source: "pinned".to_string(),
};
(dec, "pinned".to_string())
}
RoutingDirective::Auto { profile } => {
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
match live.router.route(&text, prof).await {
Ok(Some(dec)) => {
let src = dec.source.clone();
(dec, src)
}
Ok(None) | Err(_) => {
tracing::warn!(
"Router is unavailable. Gracefully degrading to default model."
);
let dec = RouteDecision {
task_label: "degraded".to_string(),
complexity_score: 0.5,
complexity_tier: "degraded".to_string(),
router_request_id: None,
source: "fallback".to_string(),
};
(dec, "fallback".to_string())
}
}
}
};
// 3. Select candidates by complexity tier / pinned model
let candidates = match &req.routing {
RoutingDirective::Pinned { model_id } => vec![model_id.clone()],
RoutingDirective::Auto { .. } => {
if route_source == "fallback" {
vec![live.routing.default_model().to_string()]
} else {
live.routing.candidates(&decision.complexity_tier)
}
}
};
// Filter candidates by capabilities (e.g. tools support)
let needs_tools = !req.tools.is_empty();
let mut final_candidates = Vec::new();
for model_id in candidates {
if let Some(prov) = live.registry.get(&model_id) {
if needs_tools && !prov.caps().tools {
tracing::debug!(
"Candidate '{}' skipped: lacks support for requested tools",
model_id
);
continue;
}
final_candidates.push(model_id);
}
}
if final_candidates.is_empty() {
return Err(GatewayError::UpstreamUnavailable(
"No healthy model available that satisfies required capabilities (e.g. tools)"
.to_string(),
));
}
// 4. Attempt provider call with failover (fall back to alternatives on error)
let mut last_err = None;
let mut attempt = 0usize;
for model_id in final_candidates {
let provider = match live.registry.get(&model_id) {
Some(p) => p,
None => continue,
};
tracing::info!(
"Routing request (label: {}, tier: {}) to model '{}' via source '{}'",
decision.task_label,
decision.complexity_tier,
model_id,
route_source
);
match provider.chat(req.clone()).await {
Ok(mut resp) => {
let cost =
provider.price(resp.usage.prompt_tokens, resp.usage.completion_tokens);
resp.cortiq = RouteInfo {
task_label: decision.task_label.clone(),
complexity_score: decision.complexity_score,
complexity_tier: decision.complexity_tier.clone(),
selected_model: model_id,
route_source: route_source.clone(),
router_request_id: decision.router_request_id.clone(),
cost_usd: cost,
failover: attempt > 0,
};
return Ok(resp);
}
Err(err) => {
tracing::warn!(
"Provider '{}' call failed: {:?}. Attempting failover...",
model_id,
err
);
last_err = Some(err);
attempt += 1;
}
}
}
Err(last_err.unwrap_or_else(|| {
GatewayError::UpstreamUnavailable("All candidate models failed".to_string())
}))
}
fn extract_text(&self, messages: &[Message], strategy: &str, max_chars: usize) -> String {
let raw_text = match strategy {
"last_user" => messages
.iter()
.rfind(|m| m.role == "user")
.map(|m| m.content.clone())
.unwrap_or_default(),
"last_user_plus_system" => {
let system_msgs: Vec<String> = messages
.iter()
.filter(|m| m.role == "system")
.map(|m| m.content.clone())
.collect();
let last_user = messages
.iter()
.rfind(|m| m.role == "user")
.map(|m| m.content.as_str())
.unwrap_or("");
if system_msgs.is_empty() {
last_user.to_string()
} else {
format!("{}\n{}", system_msgs.join("\n"), last_user)
}
}
// "concat_all" and any unknown strategy fall back to concatenating all turns
_ => messages
.iter()
.map(|m| format!("{}: {}", m.role, m.content))
.collect::<Vec<String>>()
.join("\n"),
};
if raw_text.chars().count() > max_chars {
raw_text.chars().take(max_chars).collect()
} else {
raw_text
}
}
}
src/protocols/mod.rs
+17
View File
@@ -0,0 +1,17 @@
//! Incoming protocol adapters. Each one builds its own axum routes and translates
//! its protocol to/from the canonical model. Enabled via flags in `[protocols]`.
pub mod openai_chat;
use crate::state::SharedState;
use axum::Router;
/// Build the protocols router. Implemented adapters are **always** mounted;
/// each handler checks the live `protocols.*` flag from config itself, so
/// toggles in the admin panel take effect without a restart (hot switching).
pub fn build_router() -> Router<SharedState> {
Router::new().merge(openai_chat::routes())
// TODO: openai_completions, openai_embeddings, openai_models,
// anthropic_messages, mcp, native_passthrough — add here as well,
// with a check of the corresponding flag inside the handler.
}
src/protocols/openai_chat.rs
+168
View File
@@ -0,0 +1,168 @@
//! Incoming adapter for OpenAI Chat Completions: `POST /v1/chat/completions`.
//! Translates the OpenAI request body into a canonical [`ChatRequest`], passes it
//! to the pipeline, then converts the canonical [`ChatResponse`] back to OpenAI format.
use crate::error::{GatewayError, Result};
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
use crate::state::SharedState;
use axum::{extract::State, routing::post, Json, Router};
use serde::Deserialize;
pub fn routes() -> Router<SharedState> {
Router::new().route("/v1/chat/completions", post(handler))
}
#[derive(Deserialize)]
struct OpenAiChatRequest {
model: String,
messages: Vec<Message>,
#[serde(default)]
temperature: Option<f32>,
#[serde(default)]
max_tokens: Option<u32>,
#[serde(default)]
top_p: Option<f32>,
#[serde(default)]
stream: bool,
#[serde(default)]
tools: Vec<serde_json::Value>,
#[serde(flatten)]
rest: serde_json::Map<String, serde_json::Value>,
}
/// Parse the `model` field value into a routing directive.
/// `cortiq-auto[:profile]` → Auto; otherwise → Pinned(real id).
fn parse_routing(model: &str) -> RoutingDirective {
if let Some(rest) = model.strip_prefix("cortiq-auto") {
let profile = rest.strip_prefix(':').map(|p| p.to_string());
RoutingDirective::Auto { profile }
} else {
RoutingDirective::Pinned {
model_id: model.to_string(),
}
}
}
async fn handler(
State(state): State<SharedState>,
Json(req): Json<OpenAiChatRequest>,
) -> Result<impl axum::response::IntoResponse> {
// hot protocol toggle: if the adapter is disabled in config — return 404
if !state.live().cfg.protocols.openai_chat {
return Err(GatewayError::InvalidRequest(
"openai_chat protocol is disabled".into(),
));
}
if req.messages.is_empty() {
return Err(GatewayError::InvalidRequest(
"messages must not be empty".into(),
));
}
let canonical = ChatRequest {
routing: parse_routing(&req.model),
messages: req.messages,
tools: req.tools,
params: GenParams {
temperature: req.temperature,
max_tokens: req.max_tokens,
top_p: req.top_p,
stop: Vec::new(),
passthrough: req.rest,
},
stream: req.stream,
meta: RequestMeta {
protocol: "openai_chat".into(),
..Default::default()
},
};
if req.stream {
return Err(GatewayError::InvalidRequest(
"Streaming is not yet supported in this version".into(),
));
}
let resp = state.pipeline.run(canonical, &state).await?;
let mut headers = axum::http::HeaderMap::new();
headers.insert(
"X-Cortiq-Task-Label",
axum::http::HeaderValue::from_str(&resp.cortiq.task_label)
.unwrap_or(axum::http::HeaderValue::from_static("")),
);
headers.insert(
"X-Cortiq-Complexity-Score",
axum::http::HeaderValue::from_str(&resp.cortiq.complexity_score.to_string())
.unwrap_or(axum::http::HeaderValue::from_static("0.0")),
);
headers.insert(
"X-Cortiq-Complexity-Tier",
axum::http::HeaderValue::from_str(&resp.cortiq.complexity_tier)
.unwrap_or(axum::http::HeaderValue::from_static("")),
);
headers.insert(
"X-Cortiq-Selected-Model",
axum::http::HeaderValue::from_str(&resp.cortiq.selected_model)
.unwrap_or(axum::http::HeaderValue::from_static("")),
);
headers.insert(
"X-Cortiq-Route-Source",
axum::http::HeaderValue::from_str(&resp.cortiq.route_source)
.unwrap_or(axum::http::HeaderValue::from_static("")),
);
headers.insert(
"X-Cortiq-Cost-Usd",
axum::http::HeaderValue::from_str(&resp.cortiq.cost_usd.to_string())
.unwrap_or(axum::http::HeaderValue::from_static("0.0")),
);
if let Some(req_id) = &resp.cortiq.router_request_id {
headers.insert(
"X-Cortiq-Request-Id",
axum::http::HeaderValue::from_str(req_id)
.unwrap_or(axum::http::HeaderValue::from_static("")),
);
}
let mut body = serde_json::json!({
"id": resp.id,
"object": "chat.completion",
"created": std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs(),
"model": resp.model_used,
"choices": resp.choices.iter().map(|c| {
serde_json::json!({
"index": c.index,
"message": {
"role": c.message.role,
"content": c.message.content,
"tool_calls": if c.message.tool_calls.is_empty() { serde_json::Value::Null } else { serde_json::Value::Array(c.message.tool_calls.clone()) },
},
"finish_reason": c.finish_reason,
})
}).collect::<Vec<_>>(),
"usage": {
"prompt_tokens": resp.usage.prompt_tokens,
"completion_tokens": resp.usage.completion_tokens,
"total_tokens": resp.usage.total_tokens,
}
});
if state.live().cfg.cortiq.echo {
body["cortiq"] = serde_json::json!({
"task_label": resp.cortiq.task_label,
"complexity": {
"score": resp.cortiq.complexity_score,
"tier": resp.cortiq.complexity_tier,
},
"selected_model": resp.cortiq.selected_model,
"route_source": resp.cortiq.route_source,
"router_request_id": resp.cortiq.router_request_id,
"cost_usd": resp.cortiq.cost_usd,
});
}
Ok((headers, Json(body)))
}
src/providers/anthropic.rs
+57
View File
@@ -0,0 +1,57 @@
//! Anthropic Messages provider (v0.2). Translates the canonical ChatRequest to
//! `POST /v1/messages` and back. The `system` message is extracted into a separate field.
use super::{Caps, Provider};
use crate::config::ModelCfg;
use crate::error::{GatewayError, Result};
use crate::model::{ChatRequest, ChatResponse};
use async_trait::async_trait;
pub struct AnthropicProvider {
id: String,
#[allow(dead_code)]
base_url: String,
#[allow(dead_code)]
model: String,
#[allow(dead_code)]
api_key: Option<String>,
caps: Caps,
price_in: f64,
price_out: f64,
}
impl AnthropicProvider {
pub fn new(cfg: &ModelCfg, api_key: Option<String>) -> anyhow::Result<Self> {
Ok(Self {
id: cfg.id.clone(),
base_url: cfg.base_url.trim_end_matches('/').to_string(),
model: cfg.model.clone(),
api_key,
caps: Caps::from_list(&cfg.caps),
price_in: cfg.price_in,
price_out: cfg.price_out,
})
}
}
#[async_trait]
impl Provider for AnthropicProvider {
fn id(&self) -> &str {
&self.id
}
fn caps(&self) -> &Caps {
&self.caps
}
fn price(&self, prompt_tokens: u32, completion_tokens: u32) -> f64 {
(prompt_tokens as f64 * self.price_in + completion_tokens as f64 * self.price_out)
/ 1_000_000.0
}
async fn chat(&self, _req: ChatRequest) -> Result<ChatResponse> {
// TODO(v0.2): extract system from messages → "system" field; messages → content blocks;
// add x-api-key + anthropic-version headers; map response → canonical form.
// Returns an error (not a panic) so that failover/playground continue to work normally.
Err(GatewayError::UpstreamUnavailable(
"anthropic provider not yet implemented (planned v0.2)".to_string(),
))
}
}
src/providers/mod.rs
+46
View File
@@ -0,0 +1,46 @@
//! Outgoing provider adapters. The [`Provider`] trait is the single call site for LLMs;
//! adapters translate the canonical [`ChatRequest`] to and from each provider's API format.
//!
//! Most local servers (vLLM, llama.cpp, LM Studio, Ollama `/v1`) are OpenAI-compatible
//! → [`openai::OpenAiProvider`] is sufficient for all of them.
pub mod anthropic;
pub mod openai;
use crate::model::{ChatRequest, ChatResponse};
use async_trait::async_trait;
/// Model capabilities — checked at selection time (whether tools/vision are needed).
#[derive(Clone, Debug, Default)]
pub struct Caps {
pub tools: bool,
pub vision: bool,
pub streaming: bool,
}
impl Caps {
pub fn from_list(list: &[String]) -> Self {
Self {
tools: list.iter().any(|c| c == "tools"),
vision: list.iter().any(|c| c == "vision"),
streaming: true,
}
}
}
#[async_trait]
pub trait Provider: Send + Sync {
/// The model_id from config (not the provider's model name).
fn id(&self) -> &str;
fn caps(&self) -> &Caps;
/// Cost of a request in USD — computed from price_in/price_out × tokens (for billing).
fn price(&self, prompt_tokens: u32, completion_tokens: u32) -> f64;
/// Non-streaming chat call.
async fn chat(&self, req: ChatRequest) -> crate::error::Result<ChatResponse>;
// TODO(v0.1): async fn chat_stream(...) -> Result<ChatStream> (SSE).
// TODO(v0.1): async fn embed(...) -> Result<EmbedResponse>.
}
src/providers/openai.rs
+157
View File
@@ -0,0 +1,157 @@
//! OpenAI-compatible provider. Covers OpenAI, OpenRouter, Together, Groq,
//! and local servers such as vLLM / llama.cpp / LM Studio / Ollama (`/v1`).
use super::{Caps, Provider};
use crate::config::ModelCfg;
use crate::error::{GatewayError, Result};
use crate::model::{ChatRequest, ChatResponse};
use async_trait::async_trait;
pub struct OpenAiProvider {
id: String,
base_url: String,
model: String,
api_key: Option<String>,
caps: Caps,
price_in: f64,
price_out: f64,
http: reqwest::Client,
}
impl OpenAiProvider {
pub fn new(cfg: &ModelCfg, api_key: Option<String>) -> anyhow::Result<Self> {
Ok(Self {
id: cfg.id.clone(),
base_url: cfg.base_url.trim_end_matches('/').to_string(),
model: cfg.model.clone(),
api_key,
caps: Caps::from_list(&cfg.caps),
price_in: cfg.price_in,
price_out: cfg.price_out,
http: reqwest::Client::new(),
})
}
/// Canonical request → OpenAI Chat Completions request body.
fn to_wire(&self, req: &ChatRequest) -> serde_json::Value {
let mut body = serde_json::json!({
"model": self.model,
"messages": req.messages,
"stream": req.stream,
});
if let Some(t) = req.params.temperature {
body["temperature"] = t.into();
}
if let Some(m) = req.params.max_tokens {
body["max_tokens"] = m.into();
}
if !req.tools.is_empty() {
body["tools"] = serde_json::Value::Array(req.tools.clone());
}
// uninterpreted fields are proxied as-is
for (k, v) in &req.params.passthrough {
body[k] = v.clone();
}
body
}
}
#[async_trait]
impl Provider for OpenAiProvider {
fn id(&self) -> &str {
&self.id
}
fn caps(&self) -> &Caps {
&self.caps
}
fn price(&self, prompt_tokens: u32, completion_tokens: u32) -> f64 {
(prompt_tokens as f64 * self.price_in + completion_tokens as f64 * self.price_out)
/ 1_000_000.0
}
async fn chat(&self, req: ChatRequest) -> Result<ChatResponse> {
let mut r = self
.http
.post(format!("{}/chat/completions", self.base_url))
.json(&self.to_wire(&req));
if let Some(key) = &self.api_key {
r = r.bearer_auth(key);
}
let resp = r
.send()
.await
.map_err(|e| GatewayError::UpstreamUnavailable(e.to_string()))?;
if !resp.status().is_success() {
let status = resp.status();
let err_body = resp.text().await.unwrap_or_default();
return Err(GatewayError::UpstreamUnavailable(format!(
"provider {} returned HTTP {}: {}",
self.id, status, err_body
)));
}
let v: serde_json::Value = resp
.json()
.await
.map_err(|e| GatewayError::Internal(format!("Failed to parse JSON response: {}", e)))?;
let id = v["id"].as_str().unwrap_or("").to_string();
let model_used = v["model"].as_str().unwrap_or(&self.model).to_string();
let choices_arr = v["choices"].as_array().ok_or_else(|| {
GatewayError::Internal(
"invalid openai response: choices is missing or not an array".to_string(),
)
})?;
let mut choices = Vec::new();
for (i, c) in choices_arr.iter().enumerate() {
let msg_val = &c["message"];
let role = msg_val["role"].as_str().unwrap_or("assistant").to_string();
let content = msg_val["content"].as_str().unwrap_or("").to_string();
let tool_calls = msg_val["tool_calls"]
.as_array()
.cloned()
.unwrap_or_default();
choices.push(crate::model::Choice {
index: c["index"].as_u64().unwrap_or(i as u64) as u32,
message: crate::model::Message {
role,
content,
tool_calls,
},
finish_reason: c["finish_reason"].as_str().unwrap_or("stop").to_string(),
});
}
let usage_val = &v["usage"];
let prompt_tokens = usage_val["prompt_tokens"].as_u64().unwrap_or(0) as u32;
let completion_tokens = usage_val["completion_tokens"].as_u64().unwrap_or(0) as u32;
let total_tokens = usage_val["total_tokens"]
.as_u64()
.unwrap_or((prompt_tokens + completion_tokens) as u64)
as u32;
Ok(ChatResponse {
id,
model_used,
choices,
usage: crate::model::Usage {
prompt_tokens,
completion_tokens,
total_tokens,
},
cortiq: crate::model::RouteInfo {
task_label: String::new(),
complexity_score: 0.0,
complexity_tier: String::new(),
selected_model: self.id.clone(),
route_source: String::new(),
router_request_id: None,
cost_usd: 0.0,
failover: false,
},
})
}
}
src/registry.rs
+43
View File
@@ -0,0 +1,43 @@
//! Model pool registry: builds providers from `[[models]]` and resolves
//! `model_id → Arc<dyn Provider>`.
use crate::config::Config;
use crate::providers::{anthropic::AnthropicProvider, openai::OpenAiProvider, Provider};
use crate::secrets::SecretStore;
use std::collections::HashMap;
use std::sync::Arc;
pub struct Registry {
by_id: HashMap<String, Arc<dyn Provider>>,
}
impl Registry {
pub fn from_config(cfg: &Config, secrets: &SecretStore) -> anyhow::Result<Self> {
let mut by_id: HashMap<String, Arc<dyn Provider>> = HashMap::new();
for m in &cfg.models {
if m.kind == "embedding" {
continue; // embedding models are handled by a separate branch (v0.1)
}
// provider key: secret store → environment variable
let api_key = m
.api_key_env
.as_ref()
.and_then(|name| secrets.resolve(name));
let provider: Arc<dyn Provider> = match m.provider.as_str() {
"openai" | "ollama" | "http" => Arc::new(OpenAiProvider::new(m, api_key)?),
"anthropic" => Arc::new(AnthropicProvider::new(m, api_key)?),
other => anyhow::bail!("unknown provider '{}' for model '{}'", other, m.id),
};
by_id.insert(m.id.clone(), provider);
}
Ok(Self { by_id })
}
pub fn get(&self, model_id: &str) -> Option<Arc<dyn Provider>> {
self.by_id.get(model_id).cloned()
}
pub fn ids(&self) -> impl Iterator<Item = &String> {
self.by_id.keys()
}
}
src/router_client.rs
+78
View File
@@ -0,0 +1,78 @@
//! Client for cortiq-router. Issues `POST /v1/route` and (optionally) `POST /v1/feedback`.
//! Returns the decision as a [`RouteDecision`]. When the router is unavailable the caller
//! falls back to `routing.default` (graceful degradation).
use crate::config::RouterCfg;
use crate::model::RouteDecision;
pub struct RouterClient {
http: reqwest::Client,
url: String,
api_key: Option<String>,
taxonomy_id: Option<String>,
}
impl RouterClient {
pub fn new(cfg: &RouterCfg) -> anyhow::Result<Self> {
let mut builder =
reqwest::Client::builder().timeout(std::time::Duration::from_millis(cfg.timeout_ms));
if !cfg.verify_tls {
builder = builder
.danger_accept_invalid_certs(true)
.danger_accept_invalid_hostnames(true);
}
let http = builder.build()?;
let api_key = cfg
.api_key_env
.as_ref()
.and_then(|name| std::env::var(name).ok());
Ok(Self {
http,
url: cfg.url.trim_end_matches('/').to_string(),
api_key,
taxonomy_id: cfg.taxonomy_id.clone(),
})
}
/// Classify text. Returns `Ok(None)` on network error/timeout —
/// the caller treats this as "router unavailable" and uses the default tier.
pub async fn route(&self, text: &str, profile: &str) -> anyhow::Result<Option<RouteDecision>> {
let mut body = serde_json::json!({
"input": { "text": text },
"options": { "policy_profile": profile, "allow_oracle": true }
});
if let Some(tax) = &self.taxonomy_id {
body["taxonomy_id"] = serde_json::Value::String(tax.clone());
}
let mut req = self.http.post(format!("{}/v1/route", self.url)).json(&body);
if let Some(key) = &self.api_key {
req = req.bearer_auth(key);
}
let resp = match req.send().await {
Ok(r) => r,
Err(_) => return Ok(None), // degradation
};
if !resp.status().is_success() {
return Ok(None);
}
let v: serde_json::Value = resp.json().await?;
let d = &v["decision"];
Ok(Some(RouteDecision {
task_label: d["task_label"]
.as_str()
.unwrap_or("__unknown__")
.to_string(),
complexity_score: d["complexity"]["score"].as_f64().unwrap_or(0.5) as f32,
complexity_tier: d["complexity"]["tier"]
.as_str()
.unwrap_or("medium")
.to_string(),
router_request_id: v["request_id"].as_str().map(|s| s.to_string()),
source: "router".into(),
}))
}
}
src/routing.rs
+48
View File
@@ -0,0 +1,48 @@
//! Model selection: `(task_label, complexity_tier) → ordered list of model_ids`.
//! The list serves as both a priority order and a fallback chain. See docs/ROUTING.md.
use crate::config::{Config, TierTargets};
pub struct RoutingTable {
tiers: std::collections::HashMap<String, Vec<String>>,
default: String,
}
impl RoutingTable {
pub fn from_config(cfg: &Config) -> Self {
let tiers = cfg
.routing
.tiers
.iter()
.map(|(tier, targets)| {
let list = match targets {
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
};
(tier.clone(), list)
})
.collect();
Self {
tiers,
default: cfg.routing.default.clone(),
}
}
/// Candidates for the given complexity tier, in preference order.
/// If the tier is not defined, the sole candidate is `default`.
pub fn candidates(&self, tier: &str) -> Vec<String> {
self.tiers
.get(tier)
.cloned()
.unwrap_or_else(|| vec![self.default.clone()])
}
/// Default model (used when the router is unavailable).
pub fn default_model(&self) -> &str {
&self.default
}
}
// TODO(v0.2): cost_aware mode and circuit breaker — pick the cheapest available
// model no lower than min_class[tier], respecting max_cost_usd_per_request,
// and skip models whose circuit breaker is open.
src/secrets.rs
+96
View File
@@ -0,0 +1,96 @@
//! Secret store (provider API key values) kept outside the main config.
//!
//! The file `config/secrets.toml` (in `.gitignore`, mode `0600`) is a map of
//! `env_name -> value`. Key resolution order: store entry > environment variable.
//! This allows setting keys directly from the admin panel without a restart,
//! without writing secrets into `gateway.toml`. Values are never exposed via the
//! API — only their presence status is returned (see `source`).
use std::collections::BTreeMap;
use std::sync::RwLock;
pub struct SecretStore {
path: String,
values: RwLock<BTreeMap<String, String>>,
}
impl SecretStore {
/// Load the store from a file (missing file or parse error → empty store).
pub fn load(path: &str) -> Self {
let values = std::fs::read_to_string(path)
.ok()
.and_then(|raw| toml::from_str::<BTreeMap<String, String>>(&raw).ok())
.unwrap_or_default();
Self {
path: path.to_string(),
values: RwLock::new(values),
}
}
/// Secret value by env name: store first, then environment variable.
pub fn resolve(&self, env_name: &str) -> Option<String> {
if let Some(v) = self.values.read().unwrap().get(env_name) {
if !v.is_empty() {
return Some(v.clone());
}
}
std::env::var(env_name).ok().filter(|v| !v.is_empty())
}
/// Value source for display in the UI: `store` | `env` | `missing`.
pub fn source(&self, env_name: &str) -> &'static str {
let in_store = self
.values
.read()
.unwrap()
.get(env_name)
.map(|v| !v.is_empty())
.unwrap_or(false);
if in_store {
"store"
} else if std::env::var(env_name)
.ok()
.filter(|v| !v.is_empty())
.is_some()
{
"env"
} else {
"missing"
}
}
/// Set a secret (persisted to file with mode 0600).
pub fn set(&self, env_name: &str, value: &str) -> anyhow::Result<()> {
self.values
.write()
.unwrap()
.insert(env_name.to_string(), value.to_string());
self.persist()
}
/// Remove a secret from the store (an environment variable value, if present, remains).
pub fn clear(&self, env_name: &str) -> anyhow::Result<()> {
self.values.write().unwrap().remove(env_name);
self.persist()
}
fn persist(&self) -> anyhow::Result<()> {
let data = {
let map = self.values.read().unwrap();
toml::to_string(&*map)?
};
let p = std::path::Path::new(&self.path);
if let Some(dir) = p.parent() {
std::fs::create_dir_all(dir).ok();
}
let tmp = p.with_extension("toml.tmp");
std::fs::write(&tmp, data.as_bytes())?;
std::fs::rename(&tmp, p)?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(p, std::fs::Permissions::from_mode(0o600));
}
Ok(())
}
}
src/state.rs
+107
View File
@@ -0,0 +1,107 @@
//! Shared gateway state accessible across all request handlers.
//!
//! The config-derived part (`Live`) lives behind an `ArcSwap`, which allows
//! hot-reloading the model pool, routing table, and router connection without
//! a restart: the admin API validates the new config, builds a new `Live`,
//! atomically writes the TOML, and swaps the pointer.
use crate::config::Config;
use crate::registry::Registry;
use crate::router_client::RouterClient;
use crate::routing::RoutingTable;
use crate::secrets::SecretStore;
use crate::stats::Stats;
use arc_swap::ArcSwap;
use std::sync::Arc;
#[derive(Clone)]
pub struct SharedState(pub Arc<AppState>);
/// The portion of state that is rebuilt whenever the config changes.
pub struct Live {
pub cfg: Config,
pub registry: Registry,
pub routing: RoutingTable,
pub router: RouterClient,
}
impl Live {
pub fn build(cfg: Config, secrets: &SecretStore) -> anyhow::Result<Self> {
let router = RouterClient::new(&cfg.router)?;
let registry = Registry::from_config(&cfg, secrets)?;
let routing = RoutingTable::from_config(&cfg);
Ok(Self {
cfg,
registry,
routing,
router,
})
}
}
pub struct AppState {
pub live: ArcSwap<Live>,
pub stats: Arc<Stats>,
pub secrets: SecretStore,
pub config_path: String,
pub pipeline: crate::pipeline::Pipeline,
}
impl std::ops::Deref for SharedState {
type Target = AppState;
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl SharedState {
pub fn build(cfg: Config, config_path: String) -> anyhow::Result<Self> {
let secrets_path = sibling(&config_path, "secrets.toml");
let secrets = SecretStore::load(&secrets_path);
let stats = Stats::new(&cfg.stats);
let live = Live::build(cfg, &secrets)?;
Ok(SharedState(Arc::new(AppState {
live: ArcSwap::from_pointee(live),
stats,
secrets,
config_path,
pipeline: crate::pipeline::Pipeline::new(),
})))
}
/// Current snapshot of the rebuildable state (safe to hold across await points).
pub fn live(&self) -> Arc<Live> {
self.0.live.load_full()
}
}
impl AppState {
/// Replace the config entirely: validate → build → write to disk → swap.
/// If any step fails, the active state and file are left unchanged.
pub fn reload(&self, new_cfg: Config) -> anyhow::Result<()> {
new_cfg.validate()?;
let live = Live::build(new_cfg.clone(), &self.secrets)?;
new_cfg.save(&self.config_path)?;
self.live.store(Arc::new(live));
Ok(())
}
/// Rebuild `Live` from the current config without writing to disk
/// (e.g. after a secret change — only provider keys changed).
pub fn rebuild(&self) -> anyhow::Result<()> {
let cfg = self.live.load_full().cfg.clone();
let live = Live::build(cfg, &self.secrets)?;
self.live.store(Arc::new(live));
Ok(())
}
}
/// Path to file `name` next to `path` (in the same directory).
fn sibling(path: &str, name: &str) -> String {
std::path::Path::new(path)
.parent()
.map(|d| d.join(name))
.unwrap_or_else(|| std::path::PathBuf::from(name))
.to_string_lossy()
.into_owned()
}
src/stats.rs
+395
View File
@@ -0,0 +1,395 @@
//! Request statistics tracking for the dashboard and `GET /metrics`.
//!
//! Everything is held in memory (aggregates + per-minute buckets for the retention
//! window + a ring buffer of recent requests) and optionally appended to a JSONL
//! file that is replayed on startup — so statistics survive a restart with no
//! database dependency.
use crate::config::StatsCfg;
use serde::{Deserialize, Serialize};
use std::collections::{HashMap, VecDeque};
use std::io::Write;
use std::sync::Mutex;
use std::time::{SystemTime, UNIX_EPOCH};
fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// A single record for a completed request (one JSONL line and one "recent" ring entry).
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RequestRecord {
pub ts: u64,
#[serde(default)]
pub account: String,
#[serde(default)]
pub protocol: String,
#[serde(default)]
pub directive: String, // auto | pinned
#[serde(default)]
pub task_label: String,
#[serde(default)]
pub tier: String,
#[serde(default)]
pub score: f32,
#[serde(default)]
pub model_id: String,
#[serde(default)]
pub route_source: String,
#[serde(default)]
pub prompt_tokens: u32,
#[serde(default)]
pub completion_tokens: u32,
#[serde(default)]
pub cost_usd: f64,
#[serde(default)]
pub latency_ms: u64,
#[serde(default)]
pub outcome: String, // ok | error
#[serde(default)]
pub failover: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
}
#[derive(Clone, Debug, Default, Serialize)]
pub struct Totals {
pub requests: u64,
pub ok: u64,
pub errors: u64,
pub failovers: u64,
pub prompt_tokens: u64,
pub completion_tokens: u64,
pub cost_usd: f64,
pub latency_ms_sum: u64,
}
impl Totals {
fn apply(&mut self, r: &RequestRecord) {
self.requests += 1;
if r.outcome == "ok" {
self.ok += 1;
} else {
self.errors += 1;
}
if r.failover {
self.failovers += 1;
}
self.prompt_tokens += r.prompt_tokens as u64;
self.completion_tokens += r.completion_tokens as u64;
self.cost_usd += r.cost_usd;
self.latency_ms_sum += r.latency_ms;
}
}
/// Per-minute bucket for time-series charts.
#[derive(Clone, Debug, Default, Serialize)]
pub struct Bucket {
pub minute: u64, // unix seconds, rounded down to the minute
pub requests: u64,
pub errors: u64,
pub prompt_tokens: u64,
pub completion_tokens: u64,
pub cost_usd: f64,
pub latency_ms_sum: u64,
}
#[derive(Default)]
struct Inner {
total: Totals,
by_model: HashMap<String, Totals>,
by_account: HashMap<String, Totals>,
by_tier: HashMap<String, Totals>,
by_label: HashMap<String, Totals>,
buckets: VecDeque<Bucket>,
recent: VecDeque<RequestRecord>,
}
pub struct Stats {
enabled: bool,
file: Option<String>,
ring_size: usize,
retention_secs: u64,
inner: Mutex<Inner>,
}
impl Stats {
pub fn new(cfg: &StatsCfg) -> std::sync::Arc<Self> {
let file = if cfg.file.trim().is_empty() {
None
} else {
Some(cfg.file.clone())
};
let s = Stats {
enabled: cfg.enabled,
file,
ring_size: cfg.ring_size.max(1),
retention_secs: parse_duration_secs(&cfg.retention).unwrap_or(7 * 86_400),
inner: Mutex::new(Inner::default()),
};
s.replay();
std::sync::Arc::new(s)
}
/// Replay the JSONL file into aggregates (within the retention window only).
fn replay(&self) {
let Some(path) = &self.file else { return };
let Ok(content) = std::fs::read_to_string(path) else {
return;
};
let cutoff = now_secs().saturating_sub(self.retention_secs);
let mut inner = self.inner.lock().unwrap();
for line in content.lines() {
let line = line.trim();
if line.is_empty() {
continue;
}
if let Ok(rec) = serde_json::from_str::<RequestRecord>(line) {
// push everything to the "recent" ring (it self-limits by size),
// but apply to aggregates/series only entries within the window.
Self::push_recent(&mut inner, &rec, self.ring_size);
if rec.ts >= cutoff {
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
}
}
}
}
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
inner.recent.push_back(rec.clone());
while inner.recent.len() > ring_size {
inner.recent.pop_front();
}
}
fn apply_aggregates(inner: &mut Inner, rec: &RequestRecord, retention_secs: u64) {
inner.total.apply(rec);
inner
.by_model
.entry(rec.model_id.clone())
.or_default()
.apply(rec);
inner
.by_account
.entry(if rec.account.is_empty() {
"anonymous".to_string()
} else {
rec.account.clone()
})
.or_default()
.apply(rec);
inner
.by_tier
.entry(rec.tier.clone())
.or_default()
.apply(rec);
inner
.by_label
.entry(rec.task_label.clone())
.or_default()
.apply(rec);
let minute = rec.ts - (rec.ts % 60);
match inner.buckets.back_mut() {
Some(b) if b.minute == minute => fill_bucket(b, rec),
Some(b) if b.minute > minute => { /* arrived older than the tail — ignore for series */
}
_ => {
let mut b = Bucket {
minute,
..Default::default()
};
fill_bucket(&mut b, rec);
inner.buckets.push_back(b);
}
}
// trim old buckets outside the retention window
let cutoff = now_secs().saturating_sub(retention_secs);
while let Some(front) = inner.buckets.front() {
if front.minute < cutoff {
inner.buckets.pop_front();
} else {
break;
}
}
}
/// Record a new request event.
pub fn record(&self, rec: RequestRecord) {
if !self.enabled {
return;
}
{
let mut inner = self.inner.lock().unwrap();
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
Self::push_recent(&mut inner, &rec, self.ring_size);
}
if let Some(path) = &self.file {
if let Ok(line) = serde_json::to_string(&rec) {
if let Ok(mut f) = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(path)
{
let _ = writeln!(f, "{line}");
}
}
}
}
/// Snapshot for the admin API: aggregates + time series for `range_secs` + breakdown by `groupby`.
pub fn snapshot(&self, range_secs: u64, groupby: &str) -> serde_json::Value {
let inner = self.inner.lock().unwrap();
let cutoff = now_secs().saturating_sub(range_secs);
let series: Vec<&Bucket> = inner
.buckets
.iter()
.filter(|b| b.minute >= cutoff)
.collect();
let breakdown_src = match groupby {
"account" => &inner.by_account,
"tier" => &inner.by_tier,
"label" => &inner.by_label,
_ => &inner.by_model,
};
let mut breakdown: Vec<serde_json::Value> = breakdown_src
.iter()
.map(|(k, t)| {
serde_json::json!({
"key": k,
"requests": t.requests,
"ok": t.ok,
"errors": t.errors,
"prompt_tokens": t.prompt_tokens,
"completion_tokens": t.completion_tokens,
"cost_usd": t.cost_usd,
"avg_latency_ms": avg(t.latency_ms_sum, t.requests),
})
})
.collect();
breakdown.sort_by(|a, b| {
b["requests"]
.as_u64()
.unwrap_or(0)
.cmp(&a["requests"].as_u64().unwrap_or(0))
});
serde_json::json!({
"totals": {
"requests": inner.total.requests,
"ok": inner.total.ok,
"errors": inner.total.errors,
"failovers": inner.total.failovers,
"prompt_tokens": inner.total.prompt_tokens,
"completion_tokens": inner.total.completion_tokens,
"total_tokens": inner.total.prompt_tokens + inner.total.completion_tokens,
"cost_usd": inner.total.cost_usd,
"avg_latency_ms": avg(inner.total.latency_ms_sum, inner.total.requests),
"success_rate": if inner.total.requests > 0 {
inner.total.ok as f64 / inner.total.requests as f64
} else { 0.0 },
},
"groupby": groupby,
"breakdown": breakdown,
"series": series,
})
}
/// Recent requests (newest first), with pagination.
pub fn recent(&self, limit: usize, offset: usize) -> Vec<RequestRecord> {
let inner = self.inner.lock().unwrap();
inner
.recent
.iter()
.rev()
.skip(offset)
.take(limit)
.cloned()
.collect()
}
/// Metrics text in Prometheus format.
pub fn prometheus(&self) -> String {
let inner = self.inner.lock().unwrap();
let mut out = String::new();
let t = &inner.total;
out.push_str("# HELP gw_requests_total Total gateway requests.\n");
out.push_str("# TYPE gw_requests_total counter\n");
out.push_str(&format!("gw_requests_total {}\n", t.requests));
out.push_str("# HELP gw_failovers_total Total provider failovers.\n");
out.push_str("# TYPE gw_failovers_total counter\n");
out.push_str(&format!("gw_failovers_total {}\n", t.failovers));
out.push_str("# HELP gw_tokens_total Total tokens by direction.\n");
out.push_str("# TYPE gw_tokens_total counter\n");
out.push_str(&format!(
"gw_tokens_total{{direction=\"in\"}} {}\n",
t.prompt_tokens
));
out.push_str(&format!(
"gw_tokens_total{{direction=\"out\"}} {}\n",
t.completion_tokens
));
out.push_str("# HELP gw_cost_usd_total Total estimated cost in USD.\n");
out.push_str("# TYPE gw_cost_usd_total counter\n");
out.push_str(&format!("gw_cost_usd_total {:.6}\n", t.cost_usd));
out.push_str("# HELP gw_provider_calls_total Calls by model and outcome.\n");
out.push_str("# TYPE gw_provider_calls_total counter\n");
for (model, mt) in &inner.by_model {
let model = escape_label(model);
out.push_str(&format!(
"gw_provider_calls_total{{model_id=\"{model}\",outcome=\"ok\"}} {}\n",
mt.ok
));
out.push_str(&format!(
"gw_provider_calls_total{{model_id=\"{model}\",outcome=\"error\"}} {}\n",
mt.errors
));
}
out
}
}
fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
b.requests += 1;
if r.outcome != "ok" {
b.errors += 1;
}
b.prompt_tokens += r.prompt_tokens as u64;
b.completion_tokens += r.completion_tokens as u64;
b.cost_usd += r.cost_usd;
b.latency_ms_sum += r.latency_ms;
}
fn avg(sum: u64, n: u64) -> u64 {
sum.checked_div(n).unwrap_or(0)
}
fn escape_label(s: &str) -> String {
s.replace('\\', "\\\\")
.replace('"', "\\\"")
.replace('\n', " ")
}
/// Parse a duration string of the form `7d` / `24h` / `90m` / `3600s` into seconds.
pub fn parse_duration_secs(s: &str) -> Option<u64> {
let s = s.trim();
if s.is_empty() {
return None;
}
let (num, mult) = if let Some(n) = s.strip_suffix('d') {
(n, 86_400)
} else if let Some(n) = s.strip_suffix('h') {
(n, 3_600)
} else if let Some(n) = s.strip_suffix('m') {
(n, 60)
} else if let Some(n) = s.strip_suffix('s') {
(n, 1)
} else {
(s, 1)
};
num.trim().parse::<u64>().ok().map(|v| v * mult)
}
web/css/styles.css
Vendored
+309
View File
@@ -0,0 +1,309 @@
/* ===========================================================================
allaigate Gateway Console — design system (build-free, no framework)
Ported from cortiq-console, extended with light theme, charts, KPIs, modal.
=========================================================================== */
:root,
:root[data-theme="dark"] {
--bg: #0a0e1a;
--bg-2: #0e1424;
--panel: #131a2c;
--card: #151d31;
--card-2: #121a2c;
--border: #243049;
--border-soft: #1d2740;
--text: #e6ecf7;
--muted: #8b97b0;
--faint: #5e6b85;
--accent: #6366f1;
--accent-2: #5eead4;
--gold: #f59e0b;
--good: #34d399;
--warn: #fbbf24;
--bad: #f87171;
--grid: #ffffff14;
--shadow: 0 8px 30px rgba(0, 0, 0, 0.35);
--hover: #ffffff0a;
--bg-grad: radial-gradient(1200px 700px at 80% -10%, #16203a 0%, var(--bg) 55%);
--radius: 14px;
--radius-sm: 9px;
--mono: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, monospace;
--sans: "Inter", system-ui, -apple-system, Segoe UI, Roboto, sans-serif;
}
:root[data-theme="light"] {
--bg: #f5f7fb;
--bg-2: #ffffff;
--panel: #ffffff;
--card: #ffffff;
--card-2: #f7f9fc;
--border: #d8e0ee;
--border-soft: #e6ebf4;
--text: #0f172a;
--muted: #5a6781;
--faint: #8a96ad;
--accent: #5457e6;
--accent-2: #0ea5a4;
--gold: #d97706;
--good: #10b981;
--warn: #d97706;
--bad: #ef4444;
--grid: #0f172a14;
--shadow: 0 8px 24px rgba(15, 23, 42, 0.08);
--hover: #0f172a08;
--bg-grad: radial-gradient(1200px 700px at 80% -10%, #e7edf9 0%, var(--bg) 55%);
}
* { box-sizing: border-box; }
html, body { height: 100%; }
body {
margin: 0;
font-family: var(--sans);
background: var(--bg-grad);
color: var(--text);
font-size: 14px;
-webkit-font-smoothing: antialiased;
}
a { color: var(--accent-2); text-decoration: none; }
a:hover { text-decoration: underline; }
h1, h2, h3 { font-weight: 700; letter-spacing: -0.01em; }
h2 { font-size: 20px; margin: 0 0 4px; }
h3 { font-size: 15px; margin: 0 0 10px; }
code, .mono { font-family: var(--mono); }
.boot { padding: 60px; text-align: center; color: var(--faint); }
/* ---- layout -------------------------------------------------------------- */
.app { display: grid; grid-template-columns: 250px 1fr; min-height: 100vh; }
.sidebar {
background: linear-gradient(180deg, var(--bg-2), var(--bg));
border-right: 1px solid var(--border-soft);
padding: 18px 14px;
display: flex; flex-direction: column; gap: 4px;
position: sticky; top: 0; height: 100vh;
}
.main { display: flex; flex-direction: column; min-width: 0; }
.content { padding: 24px 32px 64px; max-width: 1220px; width: 100%; }
/* ---- brand + nav --------------------------------------------------------- */
.brand { display: flex; align-items: center; gap: 10px; padding: 6px 8px 16px; }
.brand img { width: 30px; height: 30px; }
.brand b { font-size: 16px; }
.brand .sub { color: var(--muted); font-weight: 500; font-size: 13px; }
.nav-item {
display: flex; align-items: center; gap: 11px;
padding: 10px 12px; border-radius: var(--radius-sm);
color: var(--muted); cursor: pointer; font-weight: 500;
border: 1px solid transparent; user-select: none;
}
.nav-item:hover { background: var(--hover); color: var(--text); }
.nav-item.active { background: color-mix(in srgb, var(--accent) 16%, transparent); color: var(--text); border-color: color-mix(in srgb, var(--accent) 35%, transparent); }
.nav-item .ico { width: 18px; text-align: center; opacity: 0.95; font-size: 15px; }
.nav-spacer { flex: 1; }
.nav-foot { font-size: 11px; color: var(--faint); padding: 8px; }
/* ---- topbar -------------------------------------------------------------- */
.topbar {
display: flex; align-items: center; gap: 14px;
padding: 13px 32px; border-bottom: 1px solid var(--border-soft);
position: sticky; top: 0; background: color-mix(in srgb, var(--bg) 85%, transparent);
backdrop-filter: blur(8px); z-index: 5;
}
.topbar .title { font-weight: 600; font-size: 15px; }
.topbar .right { margin-left: auto; display: flex; align-items: center; gap: 10px; }
.icon-btn {
background: var(--panel); border: 1px solid var(--border); color: var(--muted);
width: 34px; height: 32px; border-radius: 8px; cursor: pointer; font-size: 15px;
display: inline-flex; align-items: center; justify-content: center;
}
.icon-btn:hover { color: var(--text); border-color: var(--accent); }
.lang-select {
background: var(--panel); border: 1px solid var(--border); color: var(--text);
border-radius: 8px; padding: 6px 8px; font-size: 12px; font-weight: 600; width: auto;
}
/* ---- status pill --------------------------------------------------------- */
.pill {
display: inline-flex; align-items: center; gap: 7px;
padding: 5px 11px; border-radius: 999px; font-size: 12px; font-weight: 600;
border: 1px solid var(--border); background: var(--panel);
}
.dot { width: 8px; height: 8px; border-radius: 50%; background: var(--faint); }
.dot.ok { background: var(--good); box-shadow: 0 0 0 3px color-mix(in srgb, var(--good) 18%, transparent); }
.dot.bad { background: var(--bad); box-shadow: 0 0 0 3px color-mix(in srgb, var(--bad) 18%, transparent); }
.dot.warn { background: var(--warn); }
/* ---- cards --------------------------------------------------------------- */
.card {
background: linear-gradient(180deg, var(--card), var(--card-2));
border: 1px solid var(--border-soft); border-radius: var(--radius);
padding: 20px; box-shadow: var(--shadow); margin-bottom: 18px;
}
.card .card-head { display: flex; align-items: center; gap: 10px; margin-bottom: 14px; }
.card .card-head .sub { color: var(--muted); font-weight: 400; font-size: 13px; }
.card .card-head .right { margin-left: auto; display: flex; gap: 8px; align-items: center; }
.grid { display: grid; gap: 18px; }
.grid.cols-2 { grid-template-columns: repeat(2, 1fr); }
.grid.cols-3 { grid-template-columns: repeat(3, 1fr); }
@media (max-width: 980px) { .grid.cols-2, .grid.cols-3 { grid-template-columns: 1fr; } .app { grid-template-columns: 1fr; } .sidebar { display: none; } }
.page-head { margin-bottom: 20px; }
.page-head .flex { justify-content: space-between; align-items: flex-start; }
.page-head p { color: var(--muted); margin: 6px 0 0; max-width: 74ch; }
/* ---- KPI ----------------------------------------------------------------- */
.kpis { display: grid; grid-template-columns: repeat(6, 1fr); gap: 14px; margin-bottom: 18px; }
@media (max-width: 1100px) { .kpis { grid-template-columns: repeat(3, 1fr); } }
@media (max-width: 620px) { .kpis { grid-template-columns: repeat(2, 1fr); } }
.kpi {
background: linear-gradient(180deg, var(--card), var(--card-2));
border: 1px solid var(--border-soft); border-radius: var(--radius); padding: 14px 16px;
}
.kpi .big { font-size: 24px; font-weight: 800; letter-spacing: -0.02em; }
.kpi .lbl { color: var(--muted); font-size: 12px; margin-top: 3px; }
/* ---- forms --------------------------------------------------------------- */
label.field { display: block; margin-bottom: 14px; }
label.field > span { display: block; color: var(--muted); font-size: 12px; margin-bottom: 6px; font-weight: 600; }
input, select, textarea {
width: 100%; background: var(--bg-2); border: 1px solid var(--border);
color: var(--text); border-radius: var(--radius-sm); padding: 10px 12px;
font-family: inherit; font-size: 14px; outline: none;
}
input:focus, select:focus, textarea:focus { border-color: var(--accent); box-shadow: 0 0 0 3px color-mix(in srgb, var(--accent) 18%, transparent); }
textarea { resize: vertical; min-height: 64px; }
.row { display: flex; gap: 12px; flex-wrap: wrap; }
.row > * { flex: 1; min-width: 150px; }
.inline { display: flex; align-items: center; gap: 8px; }
.check { display: flex; align-items: center; gap: 8px; cursor: pointer; margin-bottom: 12px; }
.check input { width: auto; }
.hint { color: var(--faint); font-size: 12px; margin-top: 4px; }
/* ---- buttons ------------------------------------------------------------- */
.btn {
display: inline-flex; align-items: center; justify-content: center; gap: 8px;
background: var(--panel); color: var(--text); border: 1px solid var(--border);
border-radius: var(--radius-sm); padding: 10px 16px; cursor: pointer;
font-weight: 600; font-size: 14px; font-family: inherit; transition: 0.12s;
}
.btn:hover { border-color: var(--accent); background: var(--hover); }
.btn:disabled { opacity: 0.5; cursor: not-allowed; }
.btn.primary { background: linear-gradient(180deg, color-mix(in srgb, var(--accent) 88%, #fff), var(--accent)); border-color: var(--accent); color: #fff; }
.btn.primary:hover { filter: brightness(1.08); }
.btn.gold { background: linear-gradient(180deg, color-mix(in srgb, var(--gold) 85%, #fff), var(--gold)); border-color: var(--gold); color: #1a1206; }
.btn.ghost { background: transparent; }
.btn.danger { color: var(--bad); border-color: color-mix(in srgb, var(--bad) 35%, transparent); }
.btn.danger:hover { background: color-mix(in srgb, var(--bad) 10%, transparent); }
.btn.sm { padding: 6px 11px; font-size: 12.5px; }
/* ---- badges -------------------------------------------------------------- */
.badge {
display: inline-flex; align-items: center; gap: 6px; padding: 3px 9px;
border-radius: 999px; font-size: 11.5px; font-weight: 700; border: 1px solid var(--border);
background: var(--bg-2); color: var(--muted);
}
.badge.tier-low, .badge.cheap, .badge.ok { color: var(--good); border-color: color-mix(in srgb, var(--good) 40%, transparent); background: color-mix(in srgb, var(--good) 8%, transparent); }
.badge.tier-medium, .badge.mid, .badge.warn { color: var(--warn); border-color: color-mix(in srgb, var(--warn) 40%, transparent); background: color-mix(in srgb, var(--warn) 8%, transparent); }
.badge.tier-high, .badge.expensive, .badge.bad, .badge.error { color: var(--bad); border-color: color-mix(in srgb, var(--bad) 40%, transparent); background: color-mix(in srgb, var(--bad) 8%, transparent); }
.badge.router, .badge.store { color: var(--accent-2); border-color: color-mix(in srgb, var(--accent-2) 40%, transparent); background: color-mix(in srgb, var(--accent-2) 8%, transparent); }
.badge.env { color: var(--accent); border-color: color-mix(in srgb, var(--accent) 40%, transparent); background: color-mix(in srgb, var(--accent) 8%, transparent); }
/* ---- tables -------------------------------------------------------------- */
.table-wrap { overflow-x: auto; }
table { width: 100%; border-collapse: collapse; }
th, td { text-align: left; padding: 11px 12px; border-bottom: 1px solid var(--border-soft); white-space: nowrap; }
th { color: var(--faint); font-size: 11px; text-transform: uppercase; letter-spacing: 0.04em; font-weight: 700; }
tr:last-child td { border-bottom: 0; }
td .mono { font-size: 12.5px; color: var(--muted); }
tbody tr:hover td { background: var(--hover); }
/* ---- chips (routing) ----------------------------------------------------- */
.tierrow { display: flex; align-items: center; gap: 12px; padding: 12px 0; border-bottom: 1px solid var(--border-soft); }
.tierrow:last-child { border-bottom: 0; }
.tierrow .tname { width: 110px; font-weight: 700; }
.chips { display: flex; gap: 8px; flex-wrap: wrap; flex: 1; align-items: center; }
.chip { display: inline-flex; align-items: center; gap: 7px; background: var(--bg-2); border: 1px solid var(--border); border-radius: 999px; padding: 5px 6px 5px 11px; font-weight: 600; font-size: 13px; }
.chip .ord { width: 16px; height: 16px; border-radius: 50%; background: var(--accent); color: #fff; font-size: 10px; display: inline-flex; align-items: center; justify-content: center; font-weight: 800; }
.chip .arrow { cursor: pointer; color: var(--faint); padding: 0 1px; }
.chip .arrow:hover { color: var(--text); }
.chip .x { cursor: pointer; color: var(--faint); border-radius: 50%; width: 18px; height: 18px; display: inline-flex; align-items: center; justify-content: center; }
.chip .x:hover { color: var(--bad); background: color-mix(in srgb, var(--bad) 12%, transparent); }
/* ---- toggle list (protocols) --------------------------------------------- */
.togglerow { display: flex; align-items: center; gap: 14px; padding: 14px 0; border-bottom: 1px solid var(--border-soft); }
.togglerow:last-child { border-bottom: 0; }
.togglerow .meta { flex: 1; }
.togglerow .meta .name { font-weight: 600; display: flex; gap: 8px; align-items: center; }
.togglerow .meta .desc { color: var(--muted); font-size: 12.5px; margin-top: 3px; }
.switch { position: relative; width: 44px; height: 24px; flex: none; }
.switch input { opacity: 0; width: 0; height: 0; }
.switch .slider { position: absolute; inset: 0; background: var(--border); border-radius: 999px; transition: 0.15s; cursor: pointer; }
.switch .slider::before { content: ""; position: absolute; width: 18px; height: 18px; left: 3px; top: 3px; background: #fff; border-radius: 50%; transition: 0.15s; }
.switch input:checked + .slider { background: var(--accent); }
.switch input:checked + .slider::before { transform: translateX(20px); }
.switch input:disabled + .slider { opacity: 0.5; cursor: not-allowed; }
/* ---- charts -------------------------------------------------------------- */
.chart { width: 100%; height: 180px; display: block; }
.chart .axis { stroke: var(--grid); stroke-width: 1; }
.chart .area { fill: color-mix(in srgb, var(--accent) 16%, transparent); }
.chart .line { fill: none; stroke: var(--accent); stroke-width: 2; }
.chart .bar { fill: var(--accent); }
.barlist { display: flex; flex-direction: column; gap: 10px; }
.barlist .b { display: grid; grid-template-columns: 130px 1fr auto; gap: 10px; align-items: center; font-size: 13px; }
.barlist .b .track { background: var(--bg-2); border-radius: 5px; height: 10px; overflow: hidden; }
.barlist .b .track > i { display: block; height: 100%; background: linear-gradient(90deg, var(--accent-2), var(--accent)); }
.barlist .b .val { color: var(--muted); font-weight: 600; min-width: 56px; text-align: right; }
/* ---- playground ---------------------------------------------------------- */
.pg { display: grid; grid-template-columns: 1fr 340px; gap: 18px; align-items: start; }
@media (max-width: 1024px) { .pg { grid-template-columns: 1fr; } }
.answer { white-space: pre-wrap; word-break: break-word; line-height: 1.55; background: var(--panel); border: 1px solid var(--border-soft); border-radius: 12px; padding: 14px 16px; min-height: 120px; }
.routecard .kv { display: flex; justify-content: space-between; gap: 12px; padding: 8px 0; border-bottom: 1px solid var(--border-soft); }
.routecard .kv:last-child { border-bottom: 0; }
.routecard .kv .k { color: var(--muted); }
.routecard .kv .v { font-weight: 600; text-align: right; }
.scorebar { height: 7px; background: var(--bg-2); border-radius: 4px; overflow: hidden; margin-top: 6px; }
.scorebar > i { display: block; height: 100%; background: linear-gradient(90deg, var(--accent-2), var(--accent)); }
/* ---- modal --------------------------------------------------------------- */
.modal-backdrop { position: fixed; inset: 0; background: rgba(3, 7, 18, 0.6); display: flex; align-items: flex-start; justify-content: center; padding: 48px 16px; z-index: 40; overflow: auto; }
.modal { background: var(--card); border: 1px solid var(--border); border-radius: var(--radius); box-shadow: var(--shadow); width: 100%; max-width: 560px; padding: 22px; }
.modal h3 { font-size: 17px; margin-bottom: 16px; }
.modal .modal-foot { display: flex; justify-content: flex-end; gap: 10px; margin-top: 8px; }
/* ---- login --------------------------------------------------------------- */
.login-wrap { min-height: 100vh; display: flex; align-items: center; justify-content: center; padding: 20px; }
.login-card { width: 100%; max-width: 400px; background: var(--card); border: 1px solid var(--border); border-radius: var(--radius); box-shadow: var(--shadow); padding: 30px; }
.login-card .brand { justify-content: center; padding-top: 0; }
.login-card h2 { text-align: center; }
.login-card p.sub { color: var(--muted); text-align: center; margin: 4px 0 22px; }
/* ---- misc ---------------------------------------------------------------- */
.empty { text-align: center; color: var(--faint); padding: 40px 20px; }
.spinner { width: 16px; height: 16px; border: 2px solid color-mix(in srgb, var(--text) 25%, transparent); border-top-color: var(--text); border-radius: 50%; animation: spin 0.7s linear infinite; display: inline-block; }
@keyframes spin { to { transform: rotate(360deg); } }
.divider { height: 1px; background: var(--border-soft); margin: 18px 0; }
.muted { color: var(--muted); }
.right-align { text-align: right; }
.flex { display: flex; gap: 10px; align-items: center; }
.flex.wrap { flex-wrap: wrap; }
.grow { flex: 1; }
.seg { display: inline-flex; border: 1px solid var(--border); border-radius: 8px; overflow: hidden; }
.seg button { background: transparent; border: 0; color: var(--muted); padding: 6px 12px; cursor: pointer; font-size: 12.5px; font-weight: 600; }
.seg button.active { background: var(--accent); color: #fff; }
/* ---- toasts -------------------------------------------------------------- */
.toasts { position: fixed; bottom: 22px; right: 22px; display: flex; flex-direction: column; gap: 10px; z-index: 60; }
.toast {
background: var(--panel); border: 1px solid var(--border); border-left: 3px solid var(--accent);
padding: 12px 16px; border-radius: 10px; box-shadow: var(--shadow); min-width: 220px;
animation: slidein 0.18s ease; font-size: 13px;
}
.toast.good { border-left-color: var(--good); }
.toast.bad { border-left-color: var(--bad); }
.toast.warn { border-left-color: var(--warn); }
@keyframes slidein { from { transform: translateX(20px); opacity: 0; } to { transform: none; opacity: 1; } }
/* ---- callout ------------------------------------------------------------- */
.callout { border: 1px solid var(--border); border-left: 3px solid var(--warn); background: color-mix(in srgb, var(--warn) 6%, transparent); border-radius: 10px; padding: 13px 15px; color: var(--muted); margin-bottom: 16px; }
.callout.info { border-left-color: var(--accent-2); background: color-mix(in srgb, var(--accent-2) 6%, transparent); }
.callout b { color: var(--text); }
web/favicon.svg
Vendored
+12
View File
@@ -0,0 +1,12 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
<defs>
<linearGradient id="g" x1="0" y1="0" x2="1" y2="1">
<stop offset="0" stop-color="#3b82f6"/>
<stop offset=".5" stop-color="#8b5cf6"/>
<stop offset="1" stop-color="#06b6d4"/>
</linearGradient>
</defs>
<rect width="32" height="32" rx="8" fill="url(#g)"/>
<path d="M9 16h14M17 10l6 6-6 6" stroke="#fff" stroke-width="2.4" fill="none"
stroke-linecap="round" stroke-linejoin="round"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 496 B

web/index.html
Vendored
+24
View File
@@ -0,0 +1,24 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>allaigate · Gateway Console</title>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="color-scheme" content="dark light" />
<link rel="icon" type="image/svg+xml" href="/admin/favicon.svg" />
<link rel="stylesheet" href="/admin/css/styles.css" />
<script>
// apply saved theme early to avoid a flash of the wrong theme
try {
var th = localStorage.getItem("allaigate_theme") || "dark";
document.documentElement.setAttribute("data-theme", th);
} catch (e) {}
</script>
</head>
<body>
<noscript>This console requires JavaScript.</noscript>
<div id="app"><div class="boot">Loading…</div></div>
<div id="toasts" class="toasts"></div>
<script type="module" src="/admin/js/app.js"></script>
</body>
</html>
web/js/api.js
Vendored
+93
View File
@@ -0,0 +1,93 @@
// Thin client over the gateway's /admin/api surface. Adds the Bearer admin token,
// parses JSON, and throws Error(message) on non-2xx so views can try/catch.
// A 401 sets an "unauthorized" flag the app uses to show the login screen.
const BASE = "/admin/api";
const TOKEN_KEY = "allaigate_token";
let onUnauthorized = null;
export function setUnauthorizedHandler(fn) {
onUnauthorized = fn;
}
export function getToken() {
return localStorage.getItem(TOKEN_KEY) || "";
}
export function setToken(tok) {
if (tok) localStorage.setItem(TOKEN_KEY, tok);
else localStorage.removeItem(TOKEN_KEY);
}
async function j(method, path, body) {
const opt = { method, headers: { Authorization: "Bearer " + getToken() } };
if (body !== undefined) {
opt.headers["content-type"] = "application/json";
opt.body = JSON.stringify(body);
}
const r = await fetch(BASE + path, opt);
if (r.status === 401) {
if (onUnauthorized) onUnauthorized();
throw new Error("unauthorized");
}
const text = await r.text();
let data = null;
try {
data = text ? JSON.parse(text) : null;
} catch {
data = text;
}
if (!r.ok) {
const msg =
(data && data.error && (data.error.message || data.error)) ||
(data && (data.detail || data.message)) ||
r.statusText;
throw new Error(typeof msg === "string" ? msg : JSON.stringify(msg));
}
return data;
}
export const api = {
// meta / health
meta: () => j("GET", "/meta"),
health: () => j("GET", "/health"),
// config
getConfig: () => j("GET", "/config"),
putConfig: (b) => j("PUT", "/config", b),
// models
listModels: () => j("GET", "/models"),
createModel: (b) => j("POST", "/models", b),
updateModel: (id, b) => j("PUT", "/models/" + encodeURIComponent(id), b),
deleteModel: (id) => j("DELETE", "/models/" + encodeURIComponent(id)),
probeModel: (id) => j("POST", "/models/" + encodeURIComponent(id) + "/probe"),
// routing
getRouting: () => j("GET", "/routing"),
putRouting: (b) => j("PUT", "/routing", b),
// protocols
getProtocols: () => j("GET", "/protocols"),
putProtocols: (b) => j("PUT", "/protocols", b),
// settings
getSettings: () => j("GET", "/settings"),
putSettings: (b) => j("PUT", "/settings", b),
// keys
listKeys: () => j("GET", "/keys"),
createKey: (b) => j("POST", "/keys", b),
deleteKey: (k) => j("DELETE", "/keys/" + encodeURIComponent(k)),
// secrets
listSecrets: () => j("GET", "/secrets"),
setSecret: (name, value) => j("PUT", "/secrets", { name, value }),
clearSecret: (name) => j("DELETE", "/secrets?name=" + encodeURIComponent(name)),
// stats
stats: (range, groupby) => j("GET", `/stats?range=${range}&groupby=${groupby}`),
requests: (limit = 50, offset = 0) => j("GET", `/requests?limit=${limit}&offset=${offset}`),
// playground
test: (b) => j("POST", "/test", b),
};
web/js/app.js
Vendored
+228
View File
@@ -0,0 +1,228 @@
// App bootstrap: login gate, sidebar, topbar, hash router, language + theme.
import { h, mount, toast } from "./ui.js";
import { t, getLang, setLang, onLangChange, LANGS } from "./i18n.js";
import { api, getToken, setToken, setUnauthorizedHandler } from "./api.js";
import { renderLogin } from "./views/login.js";
import { renderDashboard } from "./views/dashboard.js";
import { renderModels } from "./views/models.js";
import { renderRouting } from "./views/routing.js";
import { renderProtocols } from "./views/protocols.js";
import { renderKeys } from "./views/keys.js";
import { renderPlayground } from "./views/playground.js";
import { renderSettings } from "./views/settings.js";
const ROUTES = [
{ id: "dashboard", icon: "▤", view: renderDashboard },
{ id: "models", icon: "▦", view: renderModels },
{ id: "routing", icon: "⇄", view: renderRouting },
{ id: "protocols", icon: "⇆", view: renderProtocols },
{ id: "keys", icon: "🔑", view: renderKeys },
{ id: "playground", icon: "✦", view: renderPlayground },
{ id: "settings", icon: "⚙", view: renderSettings },
];
export const appState = { meta: null, health: null, version: "" };
function currentRoute() {
const id = location.hash.replace(/^#\/?/, "") || "dashboard";
return ROUTES.find((r) => r.id === id) || ROUTES[0];
}
const FAVICON =
"data:image/svg+xml;utf8," +
encodeURIComponent(
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32"><defs><linearGradient id="g" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="%233b82f6"/><stop offset=".5" stop-color="%238b5cf6"/><stop offset="1" stop-color="%2306b6d4"/></linearGradient></defs><rect width="32" height="32" rx="8" fill="url(%23g)"/><path d="M9 16h14M17 10l6 6-6 6" stroke="white" stroke-width="2.4" fill="none" stroke-linecap="round" stroke-linejoin="round"/></svg>'
);
function sidebar(activeId) {
return h(
"aside",
{ class: "sidebar", id: "sidebar" },
h(
"div",
{ class: "brand" },
h("img", { src: FAVICON, alt: "" }),
h("div", {}, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub")))
),
...ROUTES.map((r) =>
h(
"a",
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id },
h("span", { class: "ico" }, r.icon),
t("nav." + r.id)
)
),
h("div", { class: "nav-spacer" }),
h("div", { class: "nav-foot" }, t("nav.foot", { v: appState.version || "—" }))
);
}
function statusPills() {
const h_ = appState.health;
const gwOk = !!appState.meta;
const routerOk = h_?.router?.reachable;
return h(
"div",
{ class: "flex" },
h(
"span",
{ class: "pill" },
h("span", { class: "dot " + (gwOk ? "ok" : "bad") }),
t("status.gateway") + ": " + (gwOk ? t("status.online") : t("status.offline"))
),
h(
"span",
{ class: "pill", title: h_?.router?.url || "" },
h("span", { class: "dot " + (h_ ? (routerOk ? "ok" : "bad") : "warn") }),
t("status.router") + ": " + (h_ ? (routerOk ? t("status.online") : t("status.offline")) : t("status.checking"))
)
);
}
function langSelect() {
const sel = h(
"select",
{
class: "lang-select",
title: "Language",
onChange: (e) => setLang(e.target.value),
},
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, l.label))
);
return sel;
}
function themeToggle() {
const cur = document.documentElement.getAttribute("data-theme") || "dark";
return h(
"button",
{
class: "icon-btn",
title: t("theme.toggle"),
onClick: () => {
const next = (document.documentElement.getAttribute("data-theme") || "dark") === "dark" ? "light" : "dark";
document.documentElement.setAttribute("data-theme", next);
localStorage.setItem("allaigate_theme", next);
renderShell();
},
},
cur === "dark" ? "☀" : "☾"
);
}
function topbar(activeId) {
return h(
"header",
{ class: "topbar" },
h("div", { class: "title" }, t("nav." + activeId)),
h(
"div",
{ class: "right" },
statusPills(),
themeToggle(),
langSelect(),
h(
"button",
{
class: "icon-btn",
title: t("logout"),
onClick: () => {
setToken("");
boot();
},
},
"⎋"
)
)
);
}
async function renderView() {
const route = currentRoute();
const view = document.getElementById("view");
if (!view) return;
mount(view, h("div", { class: "empty" }, h("span", { class: "spinner" }), " ", t("common.loading")));
try {
const node = await route.view();
mount(view, node);
} catch (e) {
if (String(e.message) === "unauthorized") return;
mount(view, h("div", { class: "callout" }, String(e.message || e)));
}
}
function renderShell() {
const route = currentRoute();
mount(
document.getElementById("app"),
h(
"div",
{ class: "app" },
sidebar(route.id),
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" }))
)
);
renderView();
}
function showLogin() {
mount(document.getElementById("app"), renderLogin(boot));
}
async function refreshHealth() {
try {
appState.health = await api.health();
} catch {
appState.health = null;
}
// re-render topbar only if shell present
const route = currentRoute();
const main = document.querySelector(".main");
if (main) {
const old = main.querySelector(".topbar");
if (old) old.replaceWith(topbar(route.id));
}
}
export async function boot() {
// convenience: /admin?token=… logs in once, then is stripped from the URL
const urlTok = new URLSearchParams(location.search).get("token");
if (urlTok) {
setToken(urlTok);
history.replaceState({}, "", location.pathname + location.hash);
}
if (!getToken()) {
showLogin();
return;
}
try {
appState.meta = await api.meta();
appState.version = appState.meta.version || "";
} catch (e) {
if (String(e.message) === "unauthorized") {
showLogin();
return;
}
// gateway reachable but error — still show shell
}
if (!location.hash) location.hash = "#/dashboard";
renderShell();
refreshHealth();
}
setUnauthorizedHandler(() => {
toast(t("login.failed"), "bad");
setToken("");
showLogin();
});
onLangChange(() => {
if (document.querySelector(".app")) renderShell();
});
window.addEventListener("hashchange", () => {
if (document.querySelector(".app")) renderShell();
});
setInterval(() => {
if (document.querySelector(".app")) refreshHealth();
}, 20000);
boot();
web/js/charts.js
Vendored
+56
View File
@@ -0,0 +1,56 @@
// Dependency-free SVG charts. Build-free, theme-aware via CSS classes.
import { h } from "./ui.js";
const SVGNS = "http://www.w3.org/2000/svg";
function s(tag, attrs = {}, ...children) {
const el = document.createElementNS(SVGNS, tag);
for (const [k, v] of Object.entries(attrs)) if (v != null) el.setAttribute(k, v);
for (const c of children.flat()) if (c != null) el.appendChild(c);
return el;
}
// Area+line chart from time buckets. `field` is a key on each bucket.
export function lineChart(buckets, field) {
const W = 600, H = 180, padL = 8, padR = 8, padT = 12, padB = 14;
const data = (buckets || []).map((b) => Number(b[field] || 0));
const svg = s("svg", { class: "chart", viewBox: `0 0 ${W} ${H}`, preserveAspectRatio: "none" });
// baseline + mid gridlines
for (let i = 0; i <= 2; i++) {
const y = padT + ((H - padT - padB) * i) / 2;
svg.appendChild(s("line", { class: "axis", x1: padL, y1: y, x2: W - padR, y2: y }));
}
if (data.length < 2) return svg;
const max = Math.max(1, ...data);
const innerW = W - padL - padR, innerH = H - padT - padB;
const x = (i) => padL + (innerW * i) / (data.length - 1);
const y = (v) => padT + innerH - (innerH * v) / max;
let line = "", area = `M ${x(0)} ${y(0)}`;
data.forEach((v, i) => {
const cmd = i === 0 ? "M" : "L";
line += `${cmd} ${x(i).toFixed(1)} ${y(v).toFixed(1)} `;
area += ` L ${x(i).toFixed(1)} ${y(v).toFixed(1)}`;
});
area += ` L ${x(data.length - 1)} ${y(0)} Z`;
svg.appendChild(s("path", { class: "area", d: area }));
svg.appendChild(s("path", { class: "line", d: line }));
return svg;
}
// Horizontal bar list. items: [{key, value, sub}]
export function barList(items, fmt) {
const max = Math.max(1, ...items.map((i) => i.value));
return h(
"div",
{ class: "barlist" },
items.length === 0 ? h("div", { class: "empty" }, "—") : null,
...items.map((it) =>
h(
"div",
{ class: "b" },
h("div", { class: "mono", title: it.key }, it.key || "—"),
h("div", { class: "track" }, h("i", { style: `width:${(it.value / max) * 100}%` })),
h("div", { class: "val" }, fmt ? fmt(it.value) : it.value)
)
)
);
}
web/js/i18n.js
Vendored
+56
View File
@@ -0,0 +1,56 @@
// i18n engine. English is the source of truth; other languages override per key
// and fall back to English for any missing key.
import en from "./i18n/en.js";
import ru from "./i18n/ru.js";
import de from "./i18n/de.js";
import fr from "./i18n/fr.js";
import es from "./i18n/es.js";
import zh from "./i18n/zh.js";
import tr from "./i18n/tr.js";
const DICTS = { en, ru, de, fr, es, zh, tr };
export const LANGS = [
{ code: "en", label: "English" },
{ code: "ru", label: "Русский" },
{ code: "de", label: "Deutsch" },
{ code: "fr", label: "Français" },
{ code: "es", label: "Español" },
{ code: "zh", label: "中文" },
{ code: "tr", label: "Türkçe" },
];
const listeners = new Set();
function detect() {
const saved = localStorage.getItem("allaigate_lang");
if (saved && DICTS[saved]) return saved;
const url = new URLSearchParams(location.search).get("lang");
if (url && DICTS[url]) return url;
const nav = (navigator.language || "en").slice(0, 2).toLowerCase();
return DICTS[nav] ? nav : "en";
}
let lang = detect();
export function getLang() {
return lang;
}
export function setLang(l) {
lang = DICTS[l] ? l : "en";
localStorage.setItem("allaigate_lang", lang);
document.documentElement.setAttribute("lang", lang);
listeners.forEach((fn) => fn(lang));
}
export function onLangChange(fn) {
listeners.add(fn);
return () => listeners.delete(fn);
}
export function t(key, vars) {
const d = DICTS[lang] || en;
let s = d[key] ?? en[key] ?? key;
if (vars) for (const k of Object.keys(vars)) s = s.split("{" + k + "}").join(vars[k]);
return s;
}
web/js/i18n/de.js
Vendored
+256
View File
@@ -0,0 +1,256 @@
// Deutsch — German translation for allaigate admin console.
export default {
// brand / common
"brand.sub": "Gateway",
"common.loading": "Laden…",
"common.save": "Speichern",
"common.saving": "Wird gespeichert…",
"common.cancel": "Abbrechen",
"common.add": "Hinzufügen",
"common.edit": "Bearbeiten",
"common.delete": "Löschen",
"common.close": "Schließen",
"common.test": "Testen",
"common.copy": "Kopieren",
"common.copied": "Kopiert",
"common.confirm": "Bestätigen",
"common.none": "keine",
"common.saved": "Gespeichert",
"common.error": "Fehler",
"common.ok": "OK",
"common.refresh": "Aktualisieren",
"common.actions": "Aktionen",
"common.optional": "optional",
"common.search": "Suchen",
"common.all": "alle",
// nav
"nav.dashboard": "Dashboard",
"nav.models": "Modelle",
"nav.routing": "Routing",
"nav.protocols": "Protokolle",
"nav.keys": "API-Schlüssel",
"nav.playground": "Playground",
"nav.settings": "Einstellungen",
"nav.foot": "Gateway · v{v}",
// topbar / status
"status.router": "Router",
"status.gateway": "Gateway",
"status.online": "online",
"status.offline": "offline",
"status.checking": "Prüfe…",
"theme.toggle": "Design",
"logout": "Abmelden",
// login
"login.title": "allaigate Gateway",
"login.subtitle": "Admin-Token eingeben, um dieses Gateway zu verwalten.",
"login.token": "Admin-Token",
"login.tokenPlaceholder": "Admin-Token einfügen",
"login.submit": "Konsole öffnen",
"login.hint": "Das Token wird beim Start im Gateway-Log ausgegeben oder über [admin].token_env gesetzt.",
"login.failed": "Ungültiger Admin-Token.",
// dashboard
"dash.title": "Dashboard",
"dash.subtitle": "Live-Traffic, Kosten und Routing-Analyse für Ihren Modell-Pool.",
"dash.range": "Zeitraum",
"dash.range.1h": "1h",
"dash.range.24h": "24h",
"dash.range.7d": "7d",
"dash.kpi.requests": "Anfragen",
"dash.kpi.tokens": "Tokens",
"dash.kpi.cost": "Kosten",
"dash.kpi.latency": "Ø Latenz",
"dash.kpi.success": "Erfolgsrate",
"dash.kpi.failovers": "Failovers",
"dash.series.title": "Anfragen im Zeitverlauf",
"dash.series.cost": "Kosten im Zeitverlauf",
"dash.series.empty": "Kein Traffic in diesem Zeitraum.",
"dash.breakdown.title": "Aufschlüsselung",
"dash.groupby.model": "nach Modell",
"dash.groupby.tier": "nach Tier",
"dash.groupby.label": "nach Aufgabe",
"dash.groupby.account": "nach Account",
"dash.recent.title": "Letzte Anfragen",
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Playground ausprobieren.",
"dash.col.time": "Zeit",
"dash.col.model": "Modell",
"dash.col.tier": "Tier",
"dash.col.task": "Aufgabe",
"dash.col.tokens": "Tokens",
"dash.col.cost": "Kosten",
"dash.col.latency": "Latenz",
"dash.col.outcome": "Ergebnis",
"dash.col.requests": "Anfragen",
"dash.col.errors": "Fehler",
"dash.health.title": "Status",
"dash.health.reachable": "erreichbar",
"dash.health.unreachable": "nicht erreichbar",
// models
"models.title": "Modelle",
"models.subtitle": "Ihr Modell-Pool — lokale Modelle und Cloud-APIs kombinieren.",
"models.add": "Modell hinzufügen",
"models.empty": "Noch keine Modelle. Erstes Modell hinzufügen.",
"models.col.id": "ID",
"models.col.provider": "Anbieter",
"models.col.model": "Modell",
"models.col.tier": "Kostenstufe",
"models.col.key": "Schlüssel",
"models.col.caps": "Fähigkeiten",
"models.col.price": "Preis (ein/aus)",
"models.probe": "Prüfen",
"models.probe.ok": "Erreichbar ({ms} ms)",
"models.probe.fail": "Fehlgeschlagen: {err}",
"models.form.addTitle": "Modell hinzufügen",
"models.form.editTitle": "Modell bearbeiten",
"models.form.id": "ID",
"models.form.idHint": "Eindeutige ID für das Routing, z. B. local-qwen",
"models.form.provider": "Anbieter",
"models.form.baseUrl": "Basis-URL",
"models.form.model": "Modellname",
"models.form.kind": "Typ",
"models.form.costTier": "Kostenstufe",
"models.form.priceIn": "Preis ein ($/1M)",
"models.form.priceOut": "Preis aus ($/1M)",
"models.form.apiKeyEnv": "API-Schlüssel Umgebungsvariable",
"models.form.apiKeyEnvHint": "Name der Umgebungsvariable / des gespeicherten Geheimnisses für den Anbieterschlüssel.",
"models.form.caps": "Fähigkeiten",
"models.form.secret": "Geheimwert",
"models.form.secretStored": "gespeichert",
"models.form.secretEnv": "aus Umgebung",
"models.form.secretMissing": "nicht gesetzt",
"models.form.secretPlaceholder": "Schlüssel setzen oder ersetzen (nur Schreiben)",
"models.deleteConfirm": "Modell \"{id}\" löschen?",
// routing
"routing.title": "Routing",
"routing.subtitle": "Komplexitätsstufen einer priorisierten Modellliste zuordnen (Priorität = Fallback).",
"routing.default": "Standardmodell",
"routing.defaultHint": "Wird verwendet, wenn der Router nicht erreichbar ist.",
"routing.tiers": "Stufen",
"routing.tierHint": "Chips ziehen zum Sortieren; erstes bevorzugt, die übrigen sind Fallbacks.",
"routing.addModel": "+ hinzufügen",
"routing.addTier": "Stufe hinzufügen",
"routing.tierName": "Stufenname",
"routing.policy": "Richtlinie",
"routing.policy.mode": "Modus",
"routing.policy.fixed": "Feste Tabelle",
"routing.policy.costAware": "Kostenoptimiert",
"routing.policy.maxCost": "Max. Kosten pro Anfrage ($)",
"routing.policy.minClass": "Mindestklasse je Stufe",
"routing.empty": "Keine Modelle verfügbar — zuerst Modelle hinzufügen.",
// protocols
"protocols.title": "Protokolle",
"protocols.subtitle": "Nur benötigte eingehende API-Oberflächen aktivieren. Änderungen wirken sofort.",
"protocols.implemented": "implementiert",
"protocols.planned": "geplant",
"protocols.openai_chat": "OpenAI Chat Completions",
"protocols.openai_chat.d": "POST /v1/chat/completions — kompatibel mit OpenAI SDK, LangChain, crewAI.",
"protocols.openai_completions": "OpenAI Completions (legacy)",
"protocols.openai_completions.d": "POST /v1/completions — Legacy-Integrationen.",
"protocols.openai_embeddings": "OpenAI Embeddings",
"protocols.openai_embeddings.d": "POST /v1/embeddings — RAG-Pipelines.",
"protocols.openai_models": "OpenAI Models",
"protocols.openai_models.d": "GET /v1/models — automatische Client-Erkennung.",
"protocols.anthropic_messages": "Anthropic Messages",
"protocols.anthropic_messages.d": "POST /v1/messages — Claude-native Agenten.",
"protocols.mcp": "MCP (Model Context Protocol)",
"protocols.mcp.d": "JSON-RPC / SSE — MCP-native Orchestratoren.",
"protocols.native_passthrough": "Nativer Passthrough",
"protocols.native_passthrough.d": "POST /route — direkter Zugriff auf die Routing-Entscheidung.",
// keys
"keys.title": "API-Schlüssel",
"keys.subtitle": "Virtuelle Gateway-Schlüssel für Ihre Agenten. Anbieterschlüssel verlassen das Gateway nicht.",
"keys.add": "Neuer Schlüssel",
"keys.empty": "Keine Schlüssel konfiguriert — Gateway im offenen Modus (keine Authentifizierung).",
"keys.openMode": "Offener Modus: Alle Anfragen werden akzeptiert (kein Schlüssel erforderlich).",
"keys.col.key": "Schlüssel",
"keys.col.account": "Account",
"keys.col.rate": "Rate / Min.",
"keys.col.models": "Erlaubte Modelle",
"keys.form.title": "Neuer API-Schlüssel",
"keys.form.account": "Account",
"keys.form.rate": "Ratenlimit / Min.",
"keys.form.allow": "Erlaubte Modelle",
"keys.form.allowHint": "Leer lassen, um alle Modelle zu erlauben.",
"keys.form.custom": "Eigener Schlüssel (optional)",
"keys.form.customHint": "Leer lassen für automatische Generierung sk-gw-…",
"keys.created": "Schlüssel erstellt — jetzt kopieren, er wird nicht mehr angezeigt:",
"keys.revoke": "Widerrufen",
"keys.revokeConfirm": "Diesen Schlüssel widerrufen?",
"keys.allowAll": "alle",
// playground
"pg.title": "Playground",
"pg.subtitle": "Prompt durch die Live-Pipeline senden und Routing-Entscheidung prüfen.",
"pg.system": "System-Prompt",
"pg.systemPlaceholder": "optionale Systemnachricht",
"pg.prompt": "Prompt",
"pg.promptPlaceholder": "Etwas fragen…",
"pg.mode": "Modus",
"pg.mode.auto": "Auto (Route)",
"pg.mode.pinned": "Fixiertes Modell",
"pg.profile": "Profil",
"pg.model": "Modell",
"pg.temp": "Temperatur",
"pg.maxTokens": "Max. Tokens",
"pg.send": "Senden",
"pg.sending": "Wird gesendet…",
"pg.answer": "Antwort",
"pg.empty": "Antwort und Routing-Entscheidung erscheinen hier.",
"pg.decision": "Routing-Entscheidung",
"pg.task": "Aufgabe",
"pg.tier": "Tier",
"pg.score": "Komplexität",
"pg.selected": "Gewähltes Modell",
"pg.source": "Route-Quelle",
"pg.cost": "Kosten",
"pg.latency": "Latenz",
"pg.tokens": "Tokens (ein/aus)",
"pg.failover": "Failover",
"pg.error": "Anfrage fehlgeschlagen",
// settings
"settings.title": "Einstellungen",
"settings.subtitle": "Gateway-Verbindung, Routing-Eingabe, Zuverlässigkeit und Beobachtbarkeit.",
"settings.router": "Router-Verbindung",
"settings.router.url": "Router-URL",
"settings.router.keyEnv": "Router-Schlüssel Umgebungsvariable",
"settings.router.timeout": "Timeout (ms)",
"settings.router.verifyTls": "TLS-Zertifikat prüfen",
"settings.router.taxonomy": "Taxonomie-ID",
"settings.route": "Routing-Eingabe",
"settings.route.strategy": "Textstrategie",
"settings.route.maxChars": "Max. Zeichen",
"settings.route.cacheTtl": "Cache-TTL",
"settings.route.profile": "Standardprofil",
"settings.breaker": "Circuit Breaker",
"settings.breaker.threshold": "Fehlerschwelle",
"settings.breaker.cooldown": "Cooldown",
"settings.log": "Protokollierung",
"settings.log.level": "Log-Ebene",
"settings.log.bodies": "Nachrichtentexte protokollieren (Datenschutz aus)",
"settings.telemetry": "Telemetrie",
"settings.telemetry.metrics": "Prometheus /metrics bereitstellen",
"settings.cortiq": "Antwort-Metadaten",
"settings.cortiq.echo": "cortiq-Routing-Block in Antworten einbetten",
"settings.stats": "Statistiken",
"settings.stats.enabled": "Anfrage-Statistiken aufzeichnen",
"settings.stats.file": "Statistikdatei (JSONL)",
"settings.stats.retention": "Aufbewahrung",
"settings.listen": "Lausch-Adresse",
"settings.export": "Konfiguration exportieren",
"settings.import": "Konfiguration importieren",
"settings.needsRestart": "Gespeichert. Einige Änderungen (Lausch-Adresse) erfordern einen Neustart.",
// toasts / misc
"toast.saved": "Änderungen gespeichert",
"toast.deleted": "Gelöscht",
"toast.probing": "Prüfe…",
"toast.imported": "Konfiguration importiert",
};
web/js/i18n/en.js
Vendored
+257
View File
@@ -0,0 +1,257 @@
// English — source of truth. Other languages override per key; missing keys fall
// back to English. Keep keys identical across all language files.
export default {
// brand / common
"brand.sub": "Gateway",
"common.loading": "Loading…",
"common.save": "Save",
"common.saving": "Saving…",
"common.cancel": "Cancel",
"common.add": "Add",
"common.edit": "Edit",
"common.delete": "Delete",
"common.close": "Close",
"common.test": "Test",
"common.copy": "Copy",
"common.copied": "Copied",
"common.confirm": "Confirm",
"common.none": "none",
"common.saved": "Saved",
"common.error": "Error",
"common.ok": "OK",
"common.refresh": "Refresh",
"common.actions": "Actions",
"common.optional": "optional",
"common.search": "Search",
"common.all": "all",
// nav
"nav.dashboard": "Dashboard",
"nav.models": "Models",
"nav.routing": "Routing",
"nav.protocols": "Protocols",
"nav.keys": "API keys",
"nav.playground": "Playground",
"nav.settings": "Settings",
"nav.foot": "Gateway · v{v}",
// topbar / status
"status.router": "Router",
"status.gateway": "Gateway",
"status.online": "online",
"status.offline": "offline",
"status.checking": "checking…",
"theme.toggle": "Theme",
"logout": "Log out",
// login
"login.title": "allaigate Gateway",
"login.subtitle": "Enter the admin token to manage this gateway.",
"login.token": "Admin token",
"login.tokenPlaceholder": "paste admin token",
"login.submit": "Open console",
"login.hint": "The token is printed in the gateway log on startup, or set via [admin].token_env.",
"login.failed": "Invalid admin token.",
// dashboard
"dash.title": "Dashboard",
"dash.subtitle": "Live traffic, cost and routing analytics for your model pool.",
"dash.range": "Range",
"dash.range.1h": "1h",
"dash.range.24h": "24h",
"dash.range.7d": "7d",
"dash.kpi.requests": "Requests",
"dash.kpi.tokens": "Tokens",
"dash.kpi.cost": "Cost",
"dash.kpi.latency": "Avg latency",
"dash.kpi.success": "Success rate",
"dash.kpi.failovers": "Failovers",
"dash.series.title": "Requests over time",
"dash.series.cost": "Cost over time",
"dash.series.empty": "No traffic yet in this range.",
"dash.breakdown.title": "Breakdown",
"dash.groupby.model": "by model",
"dash.groupby.tier": "by tier",
"dash.groupby.label": "by task",
"dash.groupby.account": "by account",
"dash.recent.title": "Recent requests",
"dash.recent.empty": "No requests recorded yet. Try the Playground.",
"dash.col.time": "Time",
"dash.col.model": "Model",
"dash.col.tier": "Tier",
"dash.col.task": "Task",
"dash.col.tokens": "Tokens",
"dash.col.cost": "Cost",
"dash.col.latency": "Latency",
"dash.col.outcome": "Outcome",
"dash.col.requests": "Requests",
"dash.col.errors": "Errors",
"dash.health.title": "Health",
"dash.health.reachable": "reachable",
"dash.health.unreachable": "unreachable",
// models
"models.title": "Models",
"models.subtitle": "Your model pool — mix cheap local models with cloud APIs.",
"models.add": "Add model",
"models.empty": "No models yet. Add your first model to get started.",
"models.col.id": "ID",
"models.col.provider": "Provider",
"models.col.model": "Model",
"models.col.tier": "Cost tier",
"models.col.key": "Key",
"models.col.caps": "Capabilities",
"models.col.price": "Price (in/out)",
"models.probe": "Probe",
"models.probe.ok": "Reachable ({ms} ms)",
"models.probe.fail": "Failed: {err}",
"models.form.addTitle": "Add model",
"models.form.editTitle": "Edit model",
"models.form.id": "ID",
"models.form.idHint": "Unique id used in routing, e.g. local-qwen",
"models.form.provider": "Provider",
"models.form.baseUrl": "Base URL",
"models.form.model": "Model name",
"models.form.kind": "Kind",
"models.form.costTier": "Cost tier",
"models.form.priceIn": "Price in ($/1M)",
"models.form.priceOut": "Price out ($/1M)",
"models.form.apiKeyEnv": "API key env name",
"models.form.apiKeyEnvHint": "Name of the env var / stored secret holding this provider's key.",
"models.form.caps": "Capabilities",
"models.form.secret": "Secret value",
"models.form.secretStored": "stored",
"models.form.secretEnv": "from env",
"models.form.secretMissing": "not set",
"models.form.secretPlaceholder": "set or replace key (write-only)",
"models.deleteConfirm": "Delete model \"{id}\"?",
// routing
"routing.title": "Routing",
"routing.subtitle": "Map complexity tiers to an ordered list of models (priority = fallback).",
"routing.default": "Default model",
"routing.defaultHint": "Used when the router is unreachable.",
"routing.tiers": "Tiers",
"routing.tierHint": "Drag chips to reorder; first is preferred, the rest are fallbacks.",
"routing.addModel": "+ add",
"routing.addTier": "Add tier",
"routing.tierName": "Tier name",
"routing.policy": "Policy",
"routing.policy.mode": "Mode",
"routing.policy.fixed": "Fixed table",
"routing.policy.costAware": "Cost-aware",
"routing.policy.maxCost": "Max cost per request ($)",
"routing.policy.minClass": "Minimum class per tier",
"routing.empty": "No models available — add models first.",
// protocols
"protocols.title": "Protocols",
"protocols.subtitle": "Enable only the inbound API surfaces you need. Toggles apply live.",
"protocols.implemented": "implemented",
"protocols.planned": "planned",
"protocols.openai_chat": "OpenAI Chat Completions",
"protocols.openai_chat.d": "POST /v1/chat/completions — works with OpenAI SDK, LangChain, crewAI.",
"protocols.openai_completions": "OpenAI Completions (legacy)",
"protocols.openai_completions.d": "POST /v1/completions — legacy integrations.",
"protocols.openai_embeddings": "OpenAI Embeddings",
"protocols.openai_embeddings.d": "POST /v1/embeddings — RAG pipelines.",
"protocols.openai_models": "OpenAI Models",
"protocols.openai_models.d": "GET /v1/models — client auto-discovery.",
"protocols.anthropic_messages": "Anthropic Messages",
"protocols.anthropic_messages.d": "POST /v1/messages — Claude-native agents.",
"protocols.mcp": "MCP (Model Context Protocol)",
"protocols.mcp.d": "JSON-RPC / SSE — MCP-native orchestrators.",
"protocols.native_passthrough": "Native passthrough",
"protocols.native_passthrough.d": "POST /route — direct access to the routing decision.",
// keys
"keys.title": "API keys",
"keys.subtitle": "Virtual gateway keys for your agents. Provider keys never leave the gateway.",
"keys.add": "New key",
"keys.empty": "No keys configured — the gateway is in open mode (no auth).",
"keys.openMode": "Open mode: any request is accepted (no key required).",
"keys.col.key": "Key",
"keys.col.account": "Account",
"keys.col.rate": "Rate / min",
"keys.col.models": "Allowed models",
"keys.form.title": "New API key",
"keys.form.account": "Account",
"keys.form.rate": "Rate limit / min",
"keys.form.allow": "Allowed models",
"keys.form.allowHint": "Leave empty to allow all models.",
"keys.form.custom": "Custom key (optional)",
"keys.form.customHint": "Leave empty to auto-generate sk-gw-…",
"keys.created": "Key created — copy it now, it won't be shown again:",
"keys.revoke": "Revoke",
"keys.revokeConfirm": "Revoke this key?",
"keys.allowAll": "all",
// playground
"pg.title": "Playground",
"pg.subtitle": "Send a prompt through the live pipeline and inspect the routing decision.",
"pg.system": "System prompt",
"pg.systemPlaceholder": "optional system message",
"pg.prompt": "Prompt",
"pg.promptPlaceholder": "Ask something…",
"pg.mode": "Mode",
"pg.mode.auto": "Auto (route)",
"pg.mode.pinned": "Pinned model",
"pg.profile": "Profile",
"pg.model": "Model",
"pg.temp": "Temperature",
"pg.maxTokens": "Max tokens",
"pg.send": "Send",
"pg.sending": "Sending…",
"pg.answer": "Answer",
"pg.empty": "The answer and routing decision will appear here.",
"pg.decision": "Routing decision",
"pg.task": "Task",
"pg.tier": "Tier",
"pg.score": "Complexity",
"pg.selected": "Selected model",
"pg.source": "Route source",
"pg.cost": "Cost",
"pg.latency": "Latency",
"pg.tokens": "Tokens (in/out)",
"pg.failover": "Failover",
"pg.error": "Request failed",
// settings
"settings.title": "Settings",
"settings.subtitle": "Gateway connection, routing input, reliability and observability.",
"settings.router": "Router connection",
"settings.router.url": "Router URL",
"settings.router.keyEnv": "Router key env name",
"settings.router.timeout": "Timeout (ms)",
"settings.router.verifyTls": "Verify TLS certificate",
"settings.router.taxonomy": "Taxonomy id",
"settings.route": "Routing input",
"settings.route.strategy": "Text strategy",
"settings.route.maxChars": "Max chars",
"settings.route.cacheTtl": "Cache TTL",
"settings.route.profile": "Default profile",
"settings.breaker": "Circuit breaker",
"settings.breaker.threshold": "Error threshold",
"settings.breaker.cooldown": "Cooldown",
"settings.log": "Logging",
"settings.log.level": "Log level",
"settings.log.bodies": "Log message bodies (privacy off)",
"settings.telemetry": "Telemetry",
"settings.telemetry.metrics": "Expose Prometheus /metrics",
"settings.cortiq": "Response metadata",
"settings.cortiq.echo": "Echo cortiq routing block in responses",
"settings.stats": "Statistics",
"settings.stats.enabled": "Record request statistics",
"settings.stats.file": "Stats file (JSONL)",
"settings.stats.retention": "Retention",
"settings.listen": "Listen address",
"settings.export": "Export config",
"settings.import": "Import config",
"settings.needsRestart": "Saved. Some changes (listen address) require a restart.",
// toasts / misc
"toast.saved": "Changes saved",
"toast.deleted": "Deleted",
"toast.probing": "Probing…",
"toast.imported": "Config imported",
};
web/js/i18n/es.js
Vendored
+256
View File
@@ -0,0 +1,256 @@
// Spanish — traducciones para la consola de administración de allaigate Gateway.
export default {
// brand / common
"brand.sub": "Gateway",
"common.loading": "Cargando…",
"common.save": "Guardar",
"common.saving": "Guardando…",
"common.cancel": "Cancelar",
"common.add": "Agregar",
"common.edit": "Editar",
"common.delete": "Eliminar",
"common.close": "Cerrar",
"common.test": "Probar",
"common.copy": "Copiar",
"common.copied": "Copiado",
"common.confirm": "Confirmar",
"common.none": "ninguno",
"common.saved": "Guardado",
"common.error": "Error",
"common.ok": "OK",
"common.refresh": "Actualizar",
"common.actions": "Acciones",
"common.optional": "opcional",
"common.search": "Buscar",
"common.all": "todos",
// nav
"nav.dashboard": "Panel",
"nav.models": "Modelos",
"nav.routing": "Enrutamiento",
"nav.protocols": "Protocolos",
"nav.keys": "Claves API",
"nav.playground": "Playground",
"nav.settings": "Configuración",
"nav.foot": "Gateway · v{v}",
// topbar / status
"status.router": "Router",
"status.gateway": "Gateway",
"status.online": "en línea",
"status.offline": "sin conexión",
"status.checking": "verificando…",
"theme.toggle": "Tema",
"logout": "Cerrar sesión",
// login
"login.title": "allaigate Gateway",
"login.subtitle": "Ingresa el token de administrador para gestionar este gateway.",
"login.token": "Token de administrador",
"login.tokenPlaceholder": "pega el token de administrador",
"login.submit": "Abrir consola",
"login.hint": "El token se imprime en el log del gateway al iniciar, o se configura mediante [admin].token_env.",
"login.failed": "Token de administrador inválido.",
// dashboard
"dash.title": "Panel",
"dash.subtitle": "Tráfico en vivo, costos y análisis de enrutamiento de tu pool de modelos.",
"dash.range": "Rango",
"dash.range.1h": "1h",
"dash.range.24h": "24h",
"dash.range.7d": "7d",
"dash.kpi.requests": "Solicitudes",
"dash.kpi.tokens": "Tokens",
"dash.kpi.cost": "Costo",
"dash.kpi.latency": "Latencia promedio",
"dash.kpi.success": "Tasa de éxito",
"dash.kpi.failovers": "Failovers",
"dash.series.title": "Solicitudes en el tiempo",
"dash.series.cost": "Costo en el tiempo",
"dash.series.empty": "Sin tráfico en este rango.",
"dash.breakdown.title": "Desglose",
"dash.groupby.model": "por modelo",
"dash.groupby.tier": "por nivel",
"dash.groupby.label": "por tarea",
"dash.groupby.account": "por cuenta",
"dash.recent.title": "Solicitudes recientes",
"dash.recent.empty": "Sin solicitudes registradas. Prueba el Playground.",
"dash.col.time": "Hora",
"dash.col.model": "Modelo",
"dash.col.tier": "Nivel",
"dash.col.task": "Tarea",
"dash.col.tokens": "Tokens",
"dash.col.cost": "Costo",
"dash.col.latency": "Latencia",
"dash.col.outcome": "Resultado",
"dash.col.requests": "Solicitudes",
"dash.col.errors": "Errores",
"dash.health.title": "Estado",
"dash.health.reachable": "accesible",
"dash.health.unreachable": "inaccesible",
// models
"models.title": "Modelos",
"models.subtitle": "Tu pool de modelos — combina modelos locales económicos con APIs en la nube.",
"models.add": "Agregar modelo",
"models.empty": "Sin modelos aún. Agrega tu primer modelo para comenzar.",
"models.col.id": "ID",
"models.col.provider": "Proveedor",
"models.col.model": "Modelo",
"models.col.tier": "Nivel de costo",
"models.col.key": "Clave",
"models.col.caps": "Capacidades",
"models.col.price": "Precio (entrada/salida)",
"models.probe": "Sondear",
"models.probe.ok": "Accesible ({ms} ms)",
"models.probe.fail": "Error: {err}",
"models.form.addTitle": "Agregar modelo",
"models.form.editTitle": "Editar modelo",
"models.form.id": "ID",
"models.form.idHint": "ID único usado en enrutamiento, ej. local-qwen",
"models.form.provider": "Proveedor",
"models.form.baseUrl": "URL base",
"models.form.model": "Nombre del modelo",
"models.form.kind": "Tipo",
"models.form.costTier": "Nivel de costo",
"models.form.priceIn": "Precio entrada ($/1M)",
"models.form.priceOut": "Precio salida ($/1M)",
"models.form.apiKeyEnv": "Nombre de variable env para clave API",
"models.form.apiKeyEnvHint": "Nombre de la variable de entorno o secreto que contiene la clave del proveedor.",
"models.form.caps": "Capacidades",
"models.form.secret": "Valor secreto",
"models.form.secretStored": "almacenado",
"models.form.secretEnv": "desde env",
"models.form.secretMissing": "no configurado",
"models.form.secretPlaceholder": "establecer o reemplazar clave (solo escritura)",
"models.deleteConfirm": "¿Eliminar modelo \"{id}\"?",
// routing
"routing.title": "Enrutamiento",
"routing.subtitle": "Asigna niveles de complejidad a una lista ordenada de modelos (prioridad = fallback).",
"routing.default": "Modelo predeterminado",
"routing.defaultHint": "Usado cuando el router no está disponible.",
"routing.tiers": "Niveles",
"routing.tierHint": "Arrastra los chips para reordenar; el primero es preferido, los demás son fallbacks.",
"routing.addModel": "+ agregar",
"routing.addTier": "Agregar nivel",
"routing.tierName": "Nombre del nivel",
"routing.policy": "Política",
"routing.policy.mode": "Modo",
"routing.policy.fixed": "Tabla fija",
"routing.policy.costAware": "Consciente del costo",
"routing.policy.maxCost": "Costo máx. por solicitud ($)",
"routing.policy.minClass": "Clase mínima por nivel",
"routing.empty": "Sin modelos disponibles — agrega modelos primero.",
// protocols
"protocols.title": "Protocolos",
"protocols.subtitle": "Activa solo las APIs de entrada que necesitas. Los cambios se aplican en vivo.",
"protocols.implemented": "implementado",
"protocols.planned": "planificado",
"protocols.openai_chat": "OpenAI Chat Completions",
"protocols.openai_chat.d": "POST /v1/chat/completions — compatible con OpenAI SDK, LangChain, crewAI.",
"protocols.openai_completions": "OpenAI Completions (legacy)",
"protocols.openai_completions.d": "POST /v1/completions — integraciones heredadas.",
"protocols.openai_embeddings": "OpenAI Embeddings",
"protocols.openai_embeddings.d": "POST /v1/embeddings — pipelines RAG.",
"protocols.openai_models": "OpenAI Models",
"protocols.openai_models.d": "GET /v1/models — autodescubrimiento de clientes.",
"protocols.anthropic_messages": "Anthropic Messages",
"protocols.anthropic_messages.d": "POST /v1/messages — agentes nativos Claude.",
"protocols.mcp": "MCP (Model Context Protocol)",
"protocols.mcp.d": "JSON-RPC / SSE — orquestadores nativos MCP.",
"protocols.native_passthrough": "Passthrough nativo",
"protocols.native_passthrough.d": "POST /route — acceso directo a la decisión de enrutamiento.",
// keys
"keys.title": "Claves API",
"keys.subtitle": "Claves de gateway virtuales para tus agentes. Las claves del proveedor nunca salen del gateway.",
"keys.add": "Nueva clave",
"keys.empty": "Sin claves configuradas — el gateway está en modo abierto (sin autenticación).",
"keys.openMode": "Modo abierto: cualquier solicitud es aceptada (no se requiere clave).",
"keys.col.key": "Clave",
"keys.col.account": "Cuenta",
"keys.col.rate": "Límite / min",
"keys.col.models": "Modelos permitidos",
"keys.form.title": "Nueva clave API",
"keys.form.account": "Cuenta",
"keys.form.rate": "Límite de tasa / min",
"keys.form.allow": "Modelos permitidos",
"keys.form.allowHint": "Deja vacío para permitir todos los modelos.",
"keys.form.custom": "Clave personalizada (opcional)",
"keys.form.customHint": "Deja vacío para generar automáticamente sk-gw-…",
"keys.created": "Clave creada — cópiala ahora, no se mostrará de nuevo:",
"keys.revoke": "Revocar",
"keys.revokeConfirm": "¿Revocar esta clave?",
"keys.allowAll": "todos",
// playground
"pg.title": "Playground",
"pg.subtitle": "Envía un prompt por el pipeline en vivo e inspecciona la decisión de enrutamiento.",
"pg.system": "Prompt del sistema",
"pg.systemPlaceholder": "mensaje de sistema opcional",
"pg.prompt": "Prompt",
"pg.promptPlaceholder": "Escribe algo…",
"pg.mode": "Modo",
"pg.mode.auto": "Auto (enrutar)",
"pg.mode.pinned": "Modelo fijo",
"pg.profile": "Perfil",
"pg.model": "Modelo",
"pg.temp": "Temperatura",
"pg.maxTokens": "Tokens máx.",
"pg.send": "Enviar",
"pg.sending": "Enviando…",
"pg.answer": "Respuesta",
"pg.empty": "La respuesta y la decisión de enrutamiento aparecerán aquí.",
"pg.decision": "Decisión de enrutamiento",
"pg.task": "Tarea",
"pg.tier": "Nivel",
"pg.score": "Complejidad",
"pg.selected": "Modelo seleccionado",
"pg.source": "Fuente de ruta",
"pg.cost": "Costo",
"pg.latency": "Latencia",
"pg.tokens": "Tokens (entrada/salida)",
"pg.failover": "Failover",
"pg.error": "Solicitud fallida",
// settings
"settings.title": "Configuración",
"settings.subtitle": "Conexión del gateway, entrada de enrutamiento, fiabilidad y observabilidad.",
"settings.router": "Conexión del router",
"settings.router.url": "URL del router",
"settings.router.keyEnv": "Nombre de variable env para clave del router",
"settings.router.timeout": "Tiempo de espera (ms)",
"settings.router.verifyTls": "Verificar certificado TLS",
"settings.router.taxonomy": "ID de taxonomía",
"settings.route": "Entrada de enrutamiento",
"settings.route.strategy": "Estrategia de texto",
"settings.route.maxChars": "Máx. caracteres",
"settings.route.cacheTtl": "TTL de caché",
"settings.route.profile": "Perfil predeterminado",
"settings.breaker": "Circuit breaker",
"settings.breaker.threshold": "Umbral de errores",
"settings.breaker.cooldown": "Cooldown",
"settings.log": "Registro",
"settings.log.level": "Nivel de log",
"settings.log.bodies": "Registrar cuerpos de mensajes (privacidad desactivada)",
"settings.telemetry": "Telemetría",
"settings.telemetry.metrics": "Exponer Prometheus /metrics",
"settings.cortiq": "Metadatos de respuesta",
"settings.cortiq.echo": "Incluir bloque de enrutamiento cortiq en respuestas",
"settings.stats": "Estadísticas",
"settings.stats.enabled": "Registrar estadísticas de solicitudes",
"settings.stats.file": "Archivo de estadísticas (JSONL)",
"settings.stats.retention": "Retención",
"settings.listen": "Dirección de escucha",
"settings.export": "Exportar configuración",
"settings.import": "Importar configuración",
"settings.needsRestart": "Guardado. Algunos cambios (dirección de escucha) requieren reinicio.",
// toasts / misc
"toast.saved": "Cambios guardados",
"toast.deleted": "Eliminado",
"toast.probing": "Sondeando…",
"toast.imported": "Configuración importada",
};
web/js/i18n/fr.js
Vendored
+256
View File
@@ -0,0 +1,256 @@
// French (fr) — same keys as en.js; missing keys fall back to English.
export default {
// brand / common
"brand.sub": "Gateway",
"common.loading": "Chargement…",
"common.save": "Enregistrer",
"common.saving": "Enregistrement…",
"common.cancel": "Annuler",
"common.add": "Ajouter",
"common.edit": "Modifier",
"common.delete": "Supprimer",
"common.close": "Fermer",
"common.test": "Tester",
"common.copy": "Copier",
"common.copied": "Copié",
"common.confirm": "Confirmer",
"common.none": "aucun",
"common.saved": "Enregistré",
"common.error": "Erreur",
"common.ok": "OK",
"common.refresh": "Actualiser",
"common.actions": "Actions",
"common.optional": "optionnel",
"common.search": "Rechercher",
"common.all": "tous",
// nav
"nav.dashboard": "Tableau de bord",
"nav.models": "Modèles",
"nav.routing": "Routage",
"nav.protocols": "Protocoles",
"nav.keys": "Clés API",
"nav.playground": "Playground",
"nav.settings": "Paramètres",
"nav.foot": "Gateway · v{v}",
// topbar / status
"status.router": "Routeur",
"status.gateway": "Gateway",
"status.online": "en ligne",
"status.offline": "hors ligne",
"status.checking": "vérification…",
"theme.toggle": "Thème",
"logout": "Déconnexion",
// login
"login.title": "allaigate Gateway",
"login.subtitle": "Saisissez le jeton administrateur pour gérer cette passerelle.",
"login.token": "Jeton admin",
"login.tokenPlaceholder": "coller le jeton admin",
"login.submit": "Ouvrir la console",
"login.hint": "Le jeton est affiché dans le log de la passerelle au démarrage, ou défini via [admin].token_env.",
"login.failed": "Jeton admin invalide.",
// dashboard
"dash.title": "Tableau de bord",
"dash.subtitle": "Trafic en direct, coûts et analyses de routage pour votre pool de modèles.",
"dash.range": "Période",
"dash.range.1h": "1h",
"dash.range.24h": "24h",
"dash.range.7d": "7j",
"dash.kpi.requests": "Requêtes",
"dash.kpi.tokens": "Tokens",
"dash.kpi.cost": "Coût",
"dash.kpi.latency": "Latence moy.",
"dash.kpi.success": "Taux de succès",
"dash.kpi.failovers": "Basculements",
"dash.series.title": "Requêtes dans le temps",
"dash.series.cost": "Coûts dans le temps",
"dash.series.empty": "Aucun trafic sur cette période.",
"dash.breakdown.title": "Répartition",
"dash.groupby.model": "par modèle",
"dash.groupby.tier": "par niveau",
"dash.groupby.label": "par tâche",
"dash.groupby.account": "par compte",
"dash.recent.title": "Requêtes récentes",
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Playground.",
"dash.col.time": "Heure",
"dash.col.model": "Modèle",
"dash.col.tier": "Niveau",
"dash.col.task": "Tâche",
"dash.col.tokens": "Tokens",
"dash.col.cost": "Coût",
"dash.col.latency": "Latence",
"dash.col.outcome": "Résultat",
"dash.col.requests": "Requêtes",
"dash.col.errors": "Erreurs",
"dash.health.title": "Santé",
"dash.health.reachable": "accessible",
"dash.health.unreachable": "inaccessible",
// models
"models.title": "Modèles",
"models.subtitle": "Votre pool de modèles — combinez modèles locaux économiques et API cloud.",
"models.add": "Ajouter un modèle",
"models.empty": "Aucun modèle. Ajoutez votre premier modèle pour commencer.",
"models.col.id": "ID",
"models.col.provider": "Fournisseur",
"models.col.model": "Modèle",
"models.col.tier": "Niveau de coût",
"models.col.key": "Clé",
"models.col.caps": "Capacités",
"models.col.price": "Prix (entrée/sortie)",
"models.probe": "Sonder",
"models.probe.ok": "Accessible ({ms} ms)",
"models.probe.fail": "Échec : {err}",
"models.form.addTitle": "Ajouter un modèle",
"models.form.editTitle": "Modifier le modèle",
"models.form.id": "ID",
"models.form.idHint": "Identifiant unique utilisé dans le routage, ex. local-qwen",
"models.form.provider": "Fournisseur",
"models.form.baseUrl": "URL de base",
"models.form.model": "Nom du modèle",
"models.form.kind": "Type",
"models.form.costTier": "Niveau de coût",
"models.form.priceIn": "Prix entrée ($/1M)",
"models.form.priceOut": "Prix sortie ($/1M)",
"models.form.apiKeyEnv": "Nom de la var. d'env. pour la clé",
"models.form.apiKeyEnvHint": "Nom de la variable d'env. ou du secret contenant la clé de ce fournisseur.",
"models.form.caps": "Capacités",
"models.form.secret": "Valeur secrète",
"models.form.secretStored": "stocké",
"models.form.secretEnv": "depuis l'env.",
"models.form.secretMissing": "non défini",
"models.form.secretPlaceholder": "définir ou remplacer la clé (écriture seule)",
"models.deleteConfirm": "Supprimer le modèle \"{id}\" ?",
// routing
"routing.title": "Routage",
"routing.subtitle": "Associez les niveaux de complexité à une liste ordonnée de modèles (priorité = repli).",
"routing.default": "Modèle par défaut",
"routing.defaultHint": "Utilisé lorsque le routeur est inaccessible.",
"routing.tiers": "Niveaux",
"routing.tierHint": "Glissez les puces pour réordonner ; le premier est préféré, les autres sont des replis.",
"routing.addModel": "+ ajouter",
"routing.addTier": "Ajouter un niveau",
"routing.tierName": "Nom du niveau",
"routing.policy": "Politique",
"routing.policy.mode": "Mode",
"routing.policy.fixed": "Table fixe",
"routing.policy.costAware": "Sensible au coût",
"routing.policy.maxCost": "Coût max par requête ($)",
"routing.policy.minClass": "Classe minimale par niveau",
"routing.empty": "Aucun modèle disponible — ajoutez des modèles d'abord.",
// protocols
"protocols.title": "Protocoles",
"protocols.subtitle": "Activez uniquement les surfaces API entrantes nécessaires. Les bascules s'appliquent en direct.",
"protocols.implemented": "implémenté",
"protocols.planned": "planifié",
"protocols.openai_chat": "OpenAI Chat Completions",
"protocols.openai_chat.d": "POST /v1/chat/completions — compatible avec OpenAI SDK, LangChain, crewAI.",
"protocols.openai_completions": "OpenAI Completions (héritage)",
"protocols.openai_completions.d": "POST /v1/completions — intégrations héritées.",
"protocols.openai_embeddings": "OpenAI Embeddings",
"protocols.openai_embeddings.d": "POST /v1/embeddings — pipelines RAG.",
"protocols.openai_models": "OpenAI Models",
"protocols.openai_models.d": "GET /v1/models — découverte automatique des clients.",
"protocols.anthropic_messages": "Anthropic Messages",
"protocols.anthropic_messages.d": "POST /v1/messages — agents natifs Claude.",
"protocols.mcp": "MCP (Model Context Protocol)",
"protocols.mcp.d": "JSON-RPC / SSE — orchestrateurs natifs MCP.",
"protocols.native_passthrough": "Passage natif",
"protocols.native_passthrough.d": "POST /route — accès direct à la décision de routage.",
// keys
"keys.title": "Clés API",
"keys.subtitle": "Clés de passerelle virtuelles pour vos agents. Les clés fournisseurs ne quittent jamais la passerelle.",
"keys.add": "Nouvelle clé",
"keys.empty": "Aucune clé configurée — la passerelle est en mode ouvert (sans auth).",
"keys.openMode": "Mode ouvert : toute requête est acceptée (aucune clé requise).",
"keys.col.key": "Clé",
"keys.col.account": "Compte",
"keys.col.rate": "Débit / min",
"keys.col.models": "Modèles autorisés",
"keys.form.title": "Nouvelle clé API",
"keys.form.account": "Compte",
"keys.form.rate": "Limite de débit / min",
"keys.form.allow": "Modèles autorisés",
"keys.form.allowHint": "Laisser vide pour autoriser tous les modèles.",
"keys.form.custom": "Clé personnalisée (optionnel)",
"keys.form.customHint": "Laisser vide pour générer automatiquement sk-gw-…",
"keys.created": "Clé créée — copiez-la maintenant, elle ne sera plus affichée :",
"keys.revoke": "Révoquer",
"keys.revokeConfirm": "Révoquer cette clé ?",
"keys.allowAll": "tous",
// playground
"pg.title": "Playground",
"pg.subtitle": "Envoyez une invite via le pipeline en direct et inspectez la décision de routage.",
"pg.system": "Invite système",
"pg.systemPlaceholder": "message système optionnel",
"pg.prompt": "Invite",
"pg.promptPlaceholder": "Posez une question…",
"pg.mode": "Mode",
"pg.mode.auto": "Auto (routage)",
"pg.mode.pinned": "Modèle épinglé",
"pg.profile": "Profil",
"pg.model": "Modèle",
"pg.temp": "Température",
"pg.maxTokens": "Tokens max",
"pg.send": "Envoyer",
"pg.sending": "Envoi…",
"pg.answer": "Réponse",
"pg.empty": "La réponse et la décision de routage s'afficheront ici.",
"pg.decision": "Décision de routage",
"pg.task": "Tâche",
"pg.tier": "Niveau",
"pg.score": "Complexité",
"pg.selected": "Modèle sélectionné",
"pg.source": "Source du routage",
"pg.cost": "Coût",
"pg.latency": "Latence",
"pg.tokens": "Tokens (entrée/sortie)",
"pg.failover": "Basculement",
"pg.error": "Requête échouée",
// settings
"settings.title": "Paramètres",
"settings.subtitle": "Connexion passerelle, entrée de routage, fiabilité et observabilité.",
"settings.router": "Connexion routeur",
"settings.router.url": "URL du routeur",
"settings.router.keyEnv": "Nom de l'env. pour la clé du routeur",
"settings.router.timeout": "Délai d'expiration (ms)",
"settings.router.verifyTls": "Vérifier le certificat TLS",
"settings.router.taxonomy": "ID de taxonomie",
"settings.route": "Entrée de routage",
"settings.route.strategy": "Stratégie de texte",
"settings.route.maxChars": "Caractères max",
"settings.route.cacheTtl": "TTL du cache",
"settings.route.profile": "Profil par défaut",
"settings.breaker": "Disjoncteur",
"settings.breaker.threshold": "Seuil d'erreur",
"settings.breaker.cooldown": "Temporisation",
"settings.log": "Journalisation",
"settings.log.level": "Niveau de log",
"settings.log.bodies": "Journaliser les corps de messages (confidentialité désactivée)",
"settings.telemetry": "Télémétrie",
"settings.telemetry.metrics": "Exposer Prometheus /metrics",
"settings.cortiq": "Métadonnées de réponse",
"settings.cortiq.echo": "Inclure le bloc de routage cortiq dans les réponses",
"settings.stats": "Statistiques",
"settings.stats.enabled": "Enregistrer les statistiques de requêtes",
"settings.stats.file": "Fichier de stats (JSONL)",
"settings.stats.retention": "Rétention",
"settings.listen": "Adresse d'écoute",
"settings.export": "Exporter la config",
"settings.import": "Importer la config",
"settings.needsRestart": "Enregistré. Certains changements (adresse d'écoute) nécessitent un redémarrage.",
// toasts / misc
"toast.saved": "Modifications enregistrées",
"toast.deleted": "Supprimé",
"toast.probing": "Sondage…",
"toast.imported": "Config importée",
};
web/js/i18n/ru.js
Vendored
+256
View File
@@ -0,0 +1,256 @@
// Russian — ru. Keys must match en.js exactly; missing keys fall back to English.
export default {
// brand / common
"brand.sub": "Шлюз",
"common.loading": "Загрузка…",
"common.save": "Сохранить",
"common.saving": "Сохранение…",
"common.cancel": "Отмена",
"common.add": "Добавить",
"common.edit": "Изменить",
"common.delete": "Удалить",
"common.close": "Закрыть",
"common.test": "Тест",
"common.copy": "Копировать",
"common.copied": "Скопировано",
"common.confirm": "Подтвердить",
"common.none": "нет",
"common.saved": "Сохранено",
"common.error": "Ошибка",
"common.ok": "OK",
"common.refresh": "Обновить",
"common.actions": "Действия",
"common.optional": "необязательно",
"common.search": "Поиск",
"common.all": "все",
// nav
"nav.dashboard": "Дашборд",
"nav.models": "Модели",
"nav.routing": "Роутинг",
"nav.protocols": "Протоколы",
"nav.keys": "API-ключи",
"nav.playground": "Playground",
"nav.settings": "Настройки",
"nav.foot": "Шлюз · v{v}",
// topbar / status
"status.router": "Роутер",
"status.gateway": "Шлюз",
"status.online": "в сети",
"status.offline": "недоступен",
"status.checking": "проверка…",
"theme.toggle": "Тема",
"logout": "Выйти",
// login
"login.title": "allaigate Gateway",
"login.subtitle": "Введите токен администратора для управления этим шлюзом.",
"login.token": "Токен администратора",
"login.tokenPlaceholder": "вставьте токен",
"login.submit": "Войти в консоль",
"login.hint": "Токен печатается в логе шлюза при запуске или задаётся через [admin].token_env.",
"login.failed": "Неверный токен администратора.",
// dashboard
"dash.title": "Дашборд",
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
"dash.range": "Период",
"dash.range.1h": "1ч",
"dash.range.24h": "24ч",
"dash.range.7d": "7д",
"dash.kpi.requests": "Запросы",
"dash.kpi.tokens": "Токены",
"dash.kpi.cost": "Стоимость",
"dash.kpi.latency": "Ср. задержка",
"dash.kpi.success": "Успешность",
"dash.kpi.failovers": "Переключения",
"dash.series.title": "Запросы по времени",
"dash.series.cost": "Расходы по времени",
"dash.series.empty": "Трафика за этот период нет.",
"dash.breakdown.title": "Разбивка",
"dash.groupby.model": "по модели",
"dash.groupby.tier": "по уровню",
"dash.groupby.label": "по задаче",
"dash.groupby.account": "по аккаунту",
"dash.recent.title": "Последние запросы",
"dash.recent.empty": "Запросов ещё нет. Попробуйте Playground.",
"dash.col.time": "Время",
"dash.col.model": "Модель",
"dash.col.tier": "Уровень",
"dash.col.task": "Задача",
"dash.col.tokens": "Токены",
"dash.col.cost": "Стоимость",
"dash.col.latency": "Задержка",
"dash.col.outcome": "Результат",
"dash.col.requests": "Запросы",
"dash.col.errors": "Ошибки",
"dash.health.title": "Состояние",
"dash.health.reachable": "доступен",
"dash.health.unreachable": "недоступен",
// models
"models.title": "Модели",
"models.subtitle": "Пул моделей — сочетайте дешёвые локальные модели с облачными API.",
"models.add": "Добавить модель",
"models.empty": "Моделей нет. Добавьте первую модель.",
"models.col.id": "ID",
"models.col.provider": "Провайдер",
"models.col.model": "Модель",
"models.col.tier": "Уровень цены",
"models.col.key": "Ключ",
"models.col.caps": "Возможности",
"models.col.price": "Цена (вх/исх)",
"models.probe": "Проверить",
"models.probe.ok": "Доступна ({ms} мс)",
"models.probe.fail": "Ошибка: {err}",
"models.form.addTitle": "Добавить модель",
"models.form.editTitle": "Редактировать модель",
"models.form.id": "ID",
"models.form.idHint": "Уникальный ID для роутинга, например local-qwen",
"models.form.provider": "Провайдер",
"models.form.baseUrl": "Base URL",
"models.form.model": "Имя модели",
"models.form.kind": "Тип",
"models.form.costTier": "Уровень цены",
"models.form.priceIn": "Цена вх. ($/1M)",
"models.form.priceOut": "Цена исх. ($/1M)",
"models.form.apiKeyEnv": "Имя env-переменной с ключом",
"models.form.apiKeyEnvHint": "Имя переменной окружения или секрета с ключом провайдера.",
"models.form.caps": "Возможности",
"models.form.secret": "Значение секрета",
"models.form.secretStored": "сохранён",
"models.form.secretEnv": "из env",
"models.form.secretMissing": "не задан",
"models.form.secretPlaceholder": "задать или заменить ключ (только запись)",
"models.deleteConfirm": "Удалить модель \"{id}\"?",
// routing
"routing.title": "Роутинг",
"routing.subtitle": "Сопоставьте уровни сложности с упорядоченным списком моделей (приоритет = fallback).",
"routing.default": "Модель по умолчанию",
"routing.defaultHint": "Используется, если роутер недоступен.",
"routing.tiers": "Уровни",
"routing.tierHint": "Перетащите чипы для изменения порядка; первый — предпочтительный, остальные — резервные.",
"routing.addModel": "+ добавить",
"routing.addTier": "Добавить уровень",
"routing.tierName": "Название уровня",
"routing.policy": "Политика",
"routing.policy.mode": "Режим",
"routing.policy.fixed": "Фиксированная таблица",
"routing.policy.costAware": "С учётом стоимости",
"routing.policy.maxCost": "Макс. стоимость запроса ($)",
"routing.policy.minClass": "Мин. класс на уровень",
"routing.empty": "Моделей нет — сначала добавьте модели.",
// protocols
"protocols.title": "Протоколы",
"protocols.subtitle": "Включайте только нужные входящие API. Переключатели применяются мгновенно.",
"protocols.implemented": "реализован",
"protocols.planned": "планируется",
"protocols.openai_chat": "OpenAI Chat Completions",
"protocols.openai_chat.d": "POST /v1/chat/completions — работает с OpenAI SDK, LangChain, crewAI.",
"protocols.openai_completions": "OpenAI Completions (устаревший)",
"protocols.openai_completions.d": "POST /v1/completions — устаревшие интеграции.",
"protocols.openai_embeddings": "OpenAI Embeddings",
"protocols.openai_embeddings.d": "POST /v1/embeddings — RAG-пайплайны.",
"protocols.openai_models": "OpenAI Models",
"protocols.openai_models.d": "GET /v1/models — автообнаружение клиентами.",
"protocols.anthropic_messages": "Anthropic Messages",
"protocols.anthropic_messages.d": "POST /v1/messages — Claude-нативные агенты.",
"protocols.mcp": "MCP (Model Context Protocol)",
"protocols.mcp.d": "JSON-RPC / SSE — MCP-нативные оркестраторы.",
"protocols.native_passthrough": "Нативный проброс",
"protocols.native_passthrough.d": "POST /route — прямой доступ к решению роутера.",
// keys
"keys.title": "API-ключи",
"keys.subtitle": "Виртуальные ключи шлюза для агентов. Ключи провайдеров не покидают шлюз.",
"keys.add": "Новый ключ",
"keys.empty": "Ключей нет — шлюз в открытом режиме (без аутентификации).",
"keys.openMode": "Открытый режим: любой запрос принимается (ключ не нужен).",
"keys.col.key": "Ключ",
"keys.col.account": "Аккаунт",
"keys.col.rate": "Лимит / мин",
"keys.col.models": "Разрешённые модели",
"keys.form.title": "Новый API-ключ",
"keys.form.account": "Аккаунт",
"keys.form.rate": "Лимит запросов / мин",
"keys.form.allow": "Разрешённые модели",
"keys.form.allowHint": "Оставьте пустым, чтобы разрешить все модели.",
"keys.form.custom": "Произвольный ключ (необязательно)",
"keys.form.customHint": "Оставьте пустым для автогенерации sk-gw-…",
"keys.created": "Ключ создан — скопируйте его сейчас, он больше не отобразится:",
"keys.revoke": "Отозвать",
"keys.revokeConfirm": "Отозвать этот ключ?",
"keys.allowAll": "все",
// playground
"pg.title": "Playground",
"pg.subtitle": "Отправьте запрос через активный пайплайн и изучите решение роутера.",
"pg.system": "Системный промпт",
"pg.systemPlaceholder": "необязательное системное сообщение",
"pg.prompt": "Промпт",
"pg.promptPlaceholder": "Задайте вопрос…",
"pg.mode": "Режим",
"pg.mode.auto": "Авто (роутинг)",
"pg.mode.pinned": "Фиксированная модель",
"pg.profile": "Профиль",
"pg.model": "Модель",
"pg.temp": "Температура",
"pg.maxTokens": "Макс. токены",
"pg.send": "Отправить",
"pg.sending": "Отправка…",
"pg.answer": "Ответ",
"pg.empty": "Ответ и решение роутера появятся здесь.",
"pg.decision": "Решение роутера",
"pg.task": "Задача",
"pg.tier": "Уровень",
"pg.score": "Сложность",
"pg.selected": "Выбранная модель",
"pg.source": "Источник маршрута",
"pg.cost": "Стоимость",
"pg.latency": "Задержка",
"pg.tokens": "Токены (вх/исх)",
"pg.failover": "Failover",
"pg.error": "Ошибка запроса",
// settings
"settings.title": "Настройки",
"settings.subtitle": "Подключение к шлюзу, входные данные роутинга, надёжность и наблюдаемость.",
"settings.router": "Подключение к роутеру",
"settings.router.url": "URL роутера",
"settings.router.keyEnv": "Имя env-переменной с ключом роутера",
"settings.router.timeout": "Таймаут (мс)",
"settings.router.verifyTls": "Проверять TLS-сертификат",
"settings.router.taxonomy": "ID таксономии",
"settings.route": "Входные данные роутинга",
"settings.route.strategy": "Стратегия текста",
"settings.route.maxChars": "Макс. символов",
"settings.route.cacheTtl": "TTL кэша",
"settings.route.profile": "Профиль по умолчанию",
"settings.breaker": "Автоматический выключатель",
"settings.breaker.threshold": "Порог ошибок",
"settings.breaker.cooldown": "Время восстановления",
"settings.log": "Логирование",
"settings.log.level": "Уровень логов",
"settings.log.bodies": "Логировать тела сообщений (приватность отключена)",
"settings.telemetry": "Телеметрия",
"settings.telemetry.metrics": "Включить Prometheus /metrics",
"settings.cortiq": "Метаданные ответа",
"settings.cortiq.echo": "Включать блок роутинга cortiq в ответы",
"settings.stats": "Статистика",
"settings.stats.enabled": "Записывать статистику запросов",
"settings.stats.file": "Файл статистики (JSONL)",
"settings.stats.retention": "Срок хранения",
"settings.listen": "Адрес прослушивания",
"settings.export": "Экспорт конфигурации",
"settings.import": "Импорт конфигурации",
"settings.needsRestart": "Сохранено. Некоторые изменения (адрес прослушивания) требуют перезапуска.",
// toasts / misc
"toast.saved": "Изменения сохранены",
"toast.deleted": "Удалено",
"toast.probing": "Проверка…",
"toast.imported": "Конфигурация импортирована",
};
web/js/i18n/tr.js
Vendored
+256
View File
@@ -0,0 +1,256 @@
// Turkish — tam çeviri; eksik anahtarlar İngilizce'ye geri döner.
export default {
// brand / common
"brand.sub": "Gateway",
"common.loading": "Yükleniyor…",
"common.save": "Kaydet",
"common.saving": "Kaydediliyor…",
"common.cancel": "İptal",
"common.add": "Ekle",
"common.edit": "Düzenle",
"common.delete": "Sil",
"common.close": "Kapat",
"common.test": "Test",
"common.copy": "Kopyala",
"common.copied": "Kopyalandı",
"common.confirm": "Onayla",
"common.none": "yok",
"common.saved": "Kaydedildi",
"common.error": "Hata",
"common.ok": "Tamam",
"common.refresh": "Yenile",
"common.actions": "İşlemler",
"common.optional": "isteğe bağlı",
"common.search": "Ara",
"common.all": "tümü",
// nav
"nav.dashboard": "Pano",
"nav.models": "Modeller",
"nav.routing": "Yönlendirme",
"nav.protocols": "Protokoller",
"nav.keys": "API anahtarları",
"nav.playground": "Oyun alanı",
"nav.settings": "Ayarlar",
"nav.foot": "Gateway · v{v}",
// topbar / status
"status.router": "Yönlendirici",
"status.gateway": "Gateway",
"status.online": "çevrimiçi",
"status.offline": "çevrimdışı",
"status.checking": "kontrol ediliyor…",
"theme.toggle": "Tema",
"logout": "Çıkış yap",
// login
"login.title": "allaigate Gateway",
"login.subtitle": "Bu gateway'i yönetmek için yönetici token'ını girin.",
"login.token": "Yönetici token'ı",
"login.tokenPlaceholder": "yönetici token'ını yapıştırın",
"login.submit": "Konsolu aç",
"login.hint": "Token, başlangıçta gateway günlüğüne yazdırılır veya [admin].token_env ile ayarlanır.",
"login.failed": "Geçersiz yönetici token'ı.",
// dashboard
"dash.title": "Pano",
"dash.subtitle": "Model havuzunuz için canlı trafik, maliyet ve yönlendirme analitiği.",
"dash.range": "Aralık",
"dash.range.1h": "1s",
"dash.range.24h": "24s",
"dash.range.7d": "7g",
"dash.kpi.requests": "İstekler",
"dash.kpi.tokens": "Token'lar",
"dash.kpi.cost": "Maliyet",
"dash.kpi.latency": "Ort. gecikme",
"dash.kpi.success": "Başarı oranı",
"dash.kpi.failovers": "Yedeklemeler",
"dash.series.title": "Zamana göre istekler",
"dash.series.cost": "Zamana göre maliyet",
"dash.series.empty": "Bu aralıkta henüz trafik yok.",
"dash.breakdown.title": "Dağılım",
"dash.groupby.model": "modele göre",
"dash.groupby.tier": "kademeye göre",
"dash.groupby.label": "göreve göre",
"dash.groupby.account": "hesaba göre",
"dash.recent.title": "Son istekler",
"dash.recent.empty": "Henüz kayıtlı istek yok. Oyun alanını deneyin.",
"dash.col.time": "Zaman",
"dash.col.model": "Model",
"dash.col.tier": "Kademe",
"dash.col.task": "Görev",
"dash.col.tokens": "Token'lar",
"dash.col.cost": "Maliyet",
"dash.col.latency": "Gecikme",
"dash.col.outcome": "Sonuç",
"dash.col.requests": "İstekler",
"dash.col.errors": "Hatalar",
"dash.health.title": "Sağlık",
"dash.health.reachable": "erişilebilir",
"dash.health.unreachable": "erişilemiyor",
// models
"models.title": "Modeller",
"models.subtitle": "Model havuzunuz — ucuz yerel modelleri bulut API'leriyle karıştırın.",
"models.add": "Model ekle",
"models.empty": "Henüz model yok. Başlamak için ilk modelinizi ekleyin.",
"models.col.id": "ID",
"models.col.provider": "Sağlayıcı",
"models.col.model": "Model",
"models.col.tier": "Maliyet kademesi",
"models.col.key": "Anahtar",
"models.col.caps": "Yetenekler",
"models.col.price": "Fiyat (giriş/çıkış)",
"models.probe": "Sına",
"models.probe.ok": "Erişilebilir ({ms} ms)",
"models.probe.fail": "Başarısız: {err}",
"models.form.addTitle": "Model ekle",
"models.form.editTitle": "Modeli düzenle",
"models.form.id": "ID",
"models.form.idHint": "Yönlendirmede kullanılan benzersiz id, örn. local-qwen",
"models.form.provider": "Sağlayıcı",
"models.form.baseUrl": "Temel URL",
"models.form.model": "Model adı",
"models.form.kind": "Tür",
"models.form.costTier": "Maliyet kademesi",
"models.form.priceIn": "Giriş fiyatı ($/1M)",
"models.form.priceOut": "Çıkış fiyatı ($/1M)",
"models.form.apiKeyEnv": "API anahtarı ortam değişkeni adı",
"models.form.apiKeyEnvHint": "Bu sağlayıcının anahtarını tutan ortam değişkeni / saklı gizli adı.",
"models.form.caps": "Yetenekler",
"models.form.secret": "Gizli değer",
"models.form.secretStored": "saklandı",
"models.form.secretEnv": "ortamdan",
"models.form.secretMissing": "ayarlanmadı",
"models.form.secretPlaceholder": "anahtarı ayarla veya değiştir (yalnızca yazma)",
"models.deleteConfirm": "\"{id}\" modelini sil?",
// routing
"routing.title": "Yönlendirme",
"routing.subtitle": "Karmaşıklık kademelerini sıralı model listesiyle eşleştirin (öncelik = yedek).",
"routing.default": "Varsayılan model",
"routing.defaultHint": "Yönlendirici erişilemez olduğunda kullanılır.",
"routing.tiers": "Kademeler",
"routing.tierHint": "Sıralamak için çipleri sürükleyin; birincisi tercih edilir, diğerleri yedektir.",
"routing.addModel": "+ ekle",
"routing.addTier": "Kademe ekle",
"routing.tierName": "Kademe adı",
"routing.policy": "İlke",
"routing.policy.mode": "Mod",
"routing.policy.fixed": "Sabit tablo",
"routing.policy.costAware": "Maliyet bilinçli",
"routing.policy.maxCost": "İstek başına maks. maliyet ($)",
"routing.policy.minClass": "Kademe başına minimum sınıf",
"routing.empty": "Kullanılabilir model yok — önce model ekleyin.",
// protocols
"protocols.title": "Protokoller",
"protocols.subtitle": "Yalnızca ihtiyaç duyduğunuz gelen API yüzeylerini etkinleştirin. Geçişler anında uygulanır.",
"protocols.implemented": "uygulandı",
"protocols.planned": "planlandı",
"protocols.openai_chat": "OpenAI Chat Completions",
"protocols.openai_chat.d": "POST /v1/chat/completions — OpenAI SDK, LangChain, crewAI ile çalışır.",
"protocols.openai_completions": "OpenAI Completions (eski)",
"protocols.openai_completions.d": "POST /v1/completions — eski entegrasyonlar.",
"protocols.openai_embeddings": "OpenAI Embeddings",
"protocols.openai_embeddings.d": "POST /v1/embeddings — RAG ardışık düzenleri.",
"protocols.openai_models": "OpenAI Models",
"protocols.openai_models.d": "GET /v1/models — istemci otomatik keşfi.",
"protocols.anthropic_messages": "Anthropic Messages",
"protocols.anthropic_messages.d": "POST /v1/messages — Claude yerel ajanları.",
"protocols.mcp": "MCP (Model Context Protocol)",
"protocols.mcp.d": "JSON-RPC / SSE — MCP yerel orkestratörler.",
"protocols.native_passthrough": "Yerel geçiş",
"protocols.native_passthrough.d": "POST /route — yönlendirme kararına doğrudan erişim.",
// keys
"keys.title": "API anahtarları",
"keys.subtitle": "Ajanlarınız için sanal gateway anahtarları. Sağlayıcı anahtarları gateway'den hiç çıkmaz.",
"keys.add": "Yeni anahtar",
"keys.empty": "Anahtar yapılandırılmadı — gateway açık modda (kimlik doğrulama yok).",
"keys.openMode": "Açık mod: tüm istekler kabul edilir (anahtar gerekmez).",
"keys.col.key": "Anahtar",
"keys.col.account": "Hesap",
"keys.col.rate": "Oran / dak",
"keys.col.models": "İzin verilen modeller",
"keys.form.title": "Yeni API anahtarı",
"keys.form.account": "Hesap",
"keys.form.rate": "Oran sınırı / dak",
"keys.form.allow": "İzin verilen modeller",
"keys.form.allowHint": "Tüm modellere izin vermek için boş bırakın.",
"keys.form.custom": "Özel anahtar (isteğe bağlı)",
"keys.form.customHint": "sk-gw-… otomatik oluşturmak için boş bırakın.",
"keys.created": "Anahtar oluşturuldu — şimdi kopyalayın, bir daha gösterilmeyecek:",
"keys.revoke": "İptal et",
"keys.revokeConfirm": "Bu anahtarı iptal et?",
"keys.allowAll": "tümü",
// playground
"pg.title": "Oyun alanı",
"pg.subtitle": "Canlı ardışık düzen üzerinden istem gönderin ve yönlendirme kararını inceleyin.",
"pg.system": "Sistem istemi",
"pg.systemPlaceholder": "isteğe bağlı sistem mesajı",
"pg.prompt": "İstem",
"pg.promptPlaceholder": "Bir şey sorun…",
"pg.mode": "Mod",
"pg.mode.auto": "Otomatik (yönlendir)",
"pg.mode.pinned": "Sabitlenmiş model",
"pg.profile": "Profil",
"pg.model": "Model",
"pg.temp": "Sıcaklık",
"pg.maxTokens": "Maks. token",
"pg.send": "Gönder",
"pg.sending": "Gönderiliyor…",
"pg.answer": "Yanıt",
"pg.empty": "Yanıt ve yönlendirme kararı burada görünecek.",
"pg.decision": "Yönlendirme kararı",
"pg.task": "Görev",
"pg.tier": "Kademe",
"pg.score": "Karmaşıklık",
"pg.selected": "Seçilen model",
"pg.source": "Rota kaynağı",
"pg.cost": "Maliyet",
"pg.latency": "Gecikme",
"pg.tokens": "Token'lar (giriş/çıkış)",
"pg.failover": "Yedekleme",
"pg.error": "İstek başarısız",
// settings
"settings.title": "Ayarlar",
"settings.subtitle": "Gateway bağlantısı, yönlendirme girişi, güvenilirlik ve gözlemlenebilirlik.",
"settings.router": "Yönlendirici bağlantısı",
"settings.router.url": "Yönlendirici URL'si",
"settings.router.keyEnv": "Yönlendirici anahtarı ortam değişkeni adı",
"settings.router.timeout": "Zaman aşımı (ms)",
"settings.router.verifyTls": "TLS sertifikasını doğrula",
"settings.router.taxonomy": "Taksonomi id",
"settings.route": "Yönlendirme girişi",
"settings.route.strategy": "Metin stratejisi",
"settings.route.maxChars": "Maks. karakter",
"settings.route.cacheTtl": "Önbellek TTL",
"settings.route.profile": "Varsayılan profil",
"settings.breaker": "Devre kesici",
"settings.breaker.threshold": "Hata eşiği",
"settings.breaker.cooldown": "Bekleme süresi",
"settings.log": "Günlükleme",
"settings.log.level": "Günlük seviyesi",
"settings.log.bodies": "Mesaj gövdelerini günlükle (gizlilik kapalı)",
"settings.telemetry": "Telemetri",
"settings.telemetry.metrics": "Prometheus /metrics'i aç",
"settings.cortiq": "Yanıt meta verisi",
"settings.cortiq.echo": "Yanıtlarda cortiq yönlendirme bloğunu yansıt",
"settings.stats": "İstatistikler",
"settings.stats.enabled": "İstek istatistiklerini kaydet",
"settings.stats.file": "İstatistik dosyası (JSONL)",
"settings.stats.retention": "Saklama süresi",
"settings.listen": "Dinleme adresi",
"settings.export": "Yapılandırmayı dışa aktar",
"settings.import": "Yapılandırmayı içe aktar",
"settings.needsRestart": "Kaydedildi. Bazı değişiklikler (dinleme adresi) yeniden başlatma gerektirir.",
// toasts / misc
"toast.saved": "Değişiklikler kaydedildi",
"toast.deleted": "Silindi",
"toast.probing": "Sınanıyor…",
"toast.imported": "Yapılandırma içe aktarıldı",
};
web/js/i18n/zh.js
Vendored
+256
View File
@@ -0,0 +1,256 @@
// 简体中文 — Simplified Chinese translation. Keys must match en.js exactly.
export default {
// brand / common
"brand.sub": "网关",
"common.loading": "加载中…",
"common.save": "保存",
"common.saving": "保存中…",
"common.cancel": "取消",
"common.add": "添加",
"common.edit": "编辑",
"common.delete": "删除",
"common.close": "关闭",
"common.test": "测试",
"common.copy": "复制",
"common.copied": "已复制",
"common.confirm": "确认",
"common.none": "无",
"common.saved": "已保存",
"common.error": "错误",
"common.ok": "确定",
"common.refresh": "刷新",
"common.actions": "操作",
"common.optional": "可选",
"common.search": "搜索",
"common.all": "全部",
// nav
"nav.dashboard": "仪表盘",
"nav.models": "模型",
"nav.routing": "路由",
"nav.protocols": "协议",
"nav.keys": "API 密钥",
"nav.playground": "沙盒",
"nav.settings": "设置",
"nav.foot": "网关 · v{v}",
// topbar / status
"status.router": "路由器",
"status.gateway": "网关",
"status.online": "在线",
"status.offline": "离线",
"status.checking": "检测中…",
"theme.toggle": "主题",
"logout": "退出",
// login
"login.title": "allaigate Gateway",
"login.subtitle": "输入管理员令牌以管理此网关。",
"login.token": "管理员令牌",
"login.tokenPlaceholder": "粘贴管理员令牌",
"login.submit": "进入控制台",
"login.hint": "令牌将在网关启动日志中打印，或通过 [admin].token_env 设置。",
"login.failed": "无效的管理员令牌。",
// dashboard
"dash.title": "仪表盘",
"dash.subtitle": "模型池的实时流量、费用与路由分析。",
"dash.range": "时间范围",
"dash.range.1h": "1小时",
"dash.range.24h": "24小时",
"dash.range.7d": "7天",
"dash.kpi.requests": "请求数",
"dash.kpi.tokens": "Token 数",
"dash.kpi.cost": "费用",
"dash.kpi.latency": "平均延迟",
"dash.kpi.success": "成功率",
"dash.kpi.failovers": "故障转移",
"dash.series.title": "请求趋势",
"dash.series.cost": "费用趋势",
"dash.series.empty": "此时间段内暂无流量。",
"dash.breakdown.title": "明细",
"dash.groupby.model": "按模型",
"dash.groupby.tier": "按层级",
"dash.groupby.label": "按任务",
"dash.groupby.account": "按账号",
"dash.recent.title": "最近请求",
"dash.recent.empty": "暂无请求记录，请前往沙盒测试。",
"dash.col.time": "时间",
"dash.col.model": "模型",
"dash.col.tier": "层级",
"dash.col.task": "任务",
"dash.col.tokens": "Token",
"dash.col.cost": "费用",
"dash.col.latency": "延迟",
"dash.col.outcome": "结果",
"dash.col.requests": "请求数",
"dash.col.errors": "错误数",
"dash.health.title": "健康状态",
"dash.health.reachable": "可达",
"dash.health.unreachable": "不可达",
// models
"models.title": "模型",
"models.subtitle": "模型池——本地低成本模型与云端 API 混合部署。",
"models.add": "添加模型",
"models.empty": "暂无模型，请添加第一个模型。",
"models.col.id": "ID",
"models.col.provider": "提供商",
"models.col.model": "模型",
"models.col.tier": "费用层级",
"models.col.key": "密钥",
"models.col.caps": "能力",
"models.col.price": "价格（输入/输出）",
"models.probe": "探测",
"models.probe.ok": "可达（{ms} ms）",
"models.probe.fail": "失败：{err}",
"models.form.addTitle": "添加模型",
"models.form.editTitle": "编辑模型",
"models.form.id": "ID",
"models.form.idHint": "路由中使用的唯一 ID，例如 local-qwen",
"models.form.provider": "提供商",
"models.form.baseUrl": "Base URL",
"models.form.model": "模型名称",
"models.form.kind": "类型",
"models.form.costTier": "费用层级",
"models.form.priceIn": "输入价格（$/1M）",
"models.form.priceOut": "输出价格（$/1M）",
"models.form.apiKeyEnv": "API 密钥环境变量名",
"models.form.apiKeyEnvHint": "持有该提供商密钥的环境变量或存储机密名称。",
"models.form.caps": "能力",
"models.form.secret": "密钥值",
"models.form.secretStored": "已存储",
"models.form.secretEnv": "来自环境变量",
"models.form.secretMissing": "未设置",
"models.form.secretPlaceholder": "设置或替换密钥（只写）",
"models.deleteConfirm": "删除模型 \"{id}\"？",
// routing
"routing.title": "路由",
"routing.subtitle": "将复杂度层级映射到有序模型列表（优先级 = 回退顺序）。",
"routing.default": "默认模型",
"routing.defaultHint": "路由器不可达时使用。",
"routing.tiers": "层级",
"routing.tierHint": "拖动标签重新排序；第一个为首选，其余为回退。",
"routing.addModel": "+ 添加",
"routing.addTier": "添加层级",
"routing.tierName": "层级名称",
"routing.policy": "策略",
"routing.policy.mode": "模式",
"routing.policy.fixed": "固定表",
"routing.policy.costAware": "成本感知",
"routing.policy.maxCost": "单请求最大费用（$）",
"routing.policy.minClass": "每层最低等级",
"routing.empty": "暂无可用模型，请先添加模型。",
// protocols
"protocols.title": "协议",
"protocols.subtitle": "仅启用所需的入站 API 接口，切换立即生效。",
"protocols.implemented": "已实现",
"protocols.planned": "计划中",
"protocols.openai_chat": "OpenAI Chat Completions",
"protocols.openai_chat.d": "POST /v1/chat/completions — 兼容 OpenAI SDK、LangChain、crewAI。",
"protocols.openai_completions": "OpenAI Completions（旧版）",
"protocols.openai_completions.d": "POST /v1/completions — 旧版集成。",
"protocols.openai_embeddings": "OpenAI Embeddings",
"protocols.openai_embeddings.d": "POST /v1/embeddings — RAG 管道。",
"protocols.openai_models": "OpenAI Models",
"protocols.openai_models.d": "GET /v1/models — 客户端自动发现。",
"protocols.anthropic_messages": "Anthropic Messages",
"protocols.anthropic_messages.d": "POST /v1/messages — Claude 原生智能体。",
"protocols.mcp": "MCP（模型上下文协议）",
"protocols.mcp.d": "JSON-RPC / SSE — MCP 原生编排器。",
"protocols.native_passthrough": "原生透传",
"protocols.native_passthrough.d": "POST /route — 直接访问路由决策。",
// keys
"keys.title": "API 密钥",
"keys.subtitle": "为您的智能体提供虚拟网关密钥，提供商密钥不离开网关。",
"keys.add": "新建密钥",
"keys.empty": "未配置密钥——网关处于开放模式（无需认证）。",
"keys.openMode": "开放模式：接受所有请求（无需密钥）。",
"keys.col.key": "密钥",
"keys.col.account": "账号",
"keys.col.rate": "速率/分钟",
"keys.col.models": "允许模型",
"keys.form.title": "新建 API 密钥",
"keys.form.account": "账号",
"keys.form.rate": "速率限制/分钟",
"keys.form.allow": "允许模型",
"keys.form.allowHint": "留空表示允许所有模型。",
"keys.form.custom": "自定义密钥（可选）",
"keys.form.customHint": "留空将自动生成 sk-gw-…",
"keys.created": "密钥已创建——请立即复制，之后将不再显示：",
"keys.revoke": "吊销",
"keys.revokeConfirm": "确认吊销此密钥？",
"keys.allowAll": "全部",
// playground
"pg.title": "沙盒",
"pg.subtitle": "通过实时管道发送提示并查看路由决策。",
"pg.system": "系统提示",
"pg.systemPlaceholder": "可选系统消息",
"pg.prompt": "提示",
"pg.promptPlaceholder": "请输入问题…",
"pg.mode": "模式",
"pg.mode.auto": "自动（路由）",
"pg.mode.pinned": "固定模型",
"pg.profile": "配置文件",
"pg.model": "模型",
"pg.temp": "温度",
"pg.maxTokens": "最大 Token 数",
"pg.send": "发送",
"pg.sending": "发送中…",
"pg.answer": "回答",
"pg.empty": "回答和路由决策将在此显示。",
"pg.decision": "路由决策",
"pg.task": "任务",
"pg.tier": "层级",
"pg.score": "复杂度",
"pg.selected": "已选模型",
"pg.source": "路由来源",
"pg.cost": "费用",
"pg.latency": "延迟",
"pg.tokens": "Token（输入/输出）",
"pg.failover": "故障转移",
"pg.error": "请求失败",
// settings
"settings.title": "设置",
"settings.subtitle": "网关连接、路由输入、可靠性与可观测性。",
"settings.router": "路由器连接",
"settings.router.url": "路由器 URL",
"settings.router.keyEnv": "路由器密钥环境变量名",
"settings.router.timeout": "超时（ms）",
"settings.router.verifyTls": "验证 TLS 证书",
"settings.router.taxonomy": "分类体系 ID",
"settings.route": "路由输入",
"settings.route.strategy": "文本策略",
"settings.route.maxChars": "最大字符数",
"settings.route.cacheTtl": "缓存 TTL",
"settings.route.profile": "默认配置文件",
"settings.breaker": "熔断器",
"settings.breaker.threshold": "错误阈值",
"settings.breaker.cooldown": "冷却时间",
"settings.log": "日志",
"settings.log.level": "日志级别",
"settings.log.bodies": "记录消息体（隐私关闭）",
"settings.telemetry": "遥测",
"settings.telemetry.metrics": "暴露 Prometheus /metrics",
"settings.cortiq": "响应元数据",
"settings.cortiq.echo": "在响应中回显 cortiq 路由块",
"settings.stats": "统计",
"settings.stats.enabled": "记录请求统计",
"settings.stats.file": "统计文件（JSONL）",
"settings.stats.retention": "保留期",
"settings.listen": "监听地址",
"settings.export": "导出配置",
"settings.import": "导入配置",
"settings.needsRestart": "已保存。部分更改（监听地址）需要重启生效。",
// toasts / misc
"toast.saved": "更改已保存",
"toast.deleted": "已删除",
"toast.probing": "探测中…",
"toast.imported": "配置已导入",
};
web/js/ui.js
Vendored
+103
View File
@@ -0,0 +1,103 @@
// Tiny DOM helpers — no framework. `h` builds elements; `mount` swaps children.
import { t } from "./i18n.js";
export function h(tag, attrs = {}, ...children) {
const el = document.createElement(tag);
for (const [k, v] of Object.entries(attrs || {})) {
if (v == null || v === false) continue;
if (k === "class") el.className = v;
else if (k === "html") el.innerHTML = v;
else if (k === "dataset") Object.assign(el.dataset, v);
else if (k.startsWith("on") && typeof v === "function") el.addEventListener(k.slice(2).toLowerCase(), v);
else if (v === true) el.setAttribute(k, "");
else el.setAttribute(k, v);
}
for (const c of children.flat()) {
if (c == null || c === false) continue;
el.appendChild(typeof c === "object" ? c : document.createTextNode(String(c)));
}
return el;
}
export function clear(node) {
while (node && node.firstChild) node.removeChild(node.firstChild);
}
export function mount(node, ...children) {
clear(node);
for (const c of children.flat()) if (c != null && c !== false) node.appendChild(c);
return node;
}
export function toast(msg, kind = "info", ms = 3200) {
const box = document.getElementById("toasts");
if (!box) return;
const el = h("div", { class: `toast ${kind}` }, msg);
box.appendChild(el);
setTimeout(() => {
el.style.opacity = "0";
el.style.transition = "opacity .2s";
setTimeout(() => el.remove(), 220);
}, ms);
}
// Modal dialog. `body` is a DOM node; `onSave` returns a Promise; returns nothing.
export function modal(title, body, onSave, saveLabel) {
const backdrop = h("div", { class: "modal-backdrop" });
const close = () => backdrop.remove();
backdrop.addEventListener("click", (e) => {
if (e.target === backdrop) close();
});
const saveBtn = h("button", { class: "btn primary" }, saveLabel || t("common.save"));
saveBtn.addEventListener("click", async () => {
saveBtn.disabled = true;
try {
const keep = await onSave();
if (keep !== false) close();
} finally {
saveBtn.disabled = false;
}
});
const card = h(
"div",
{ class: "modal" },
h("h3", {}, title),
body,
h(
"div",
{ class: "modal-foot" },
h("button", { class: "btn ghost", onClick: close }, t("common.cancel")),
onSave ? saveBtn : null
)
);
backdrop.appendChild(card);
document.body.appendChild(backdrop);
return { close };
}
export function confirmDialog(message) {
return Promise.resolve(window.confirm(message));
}
// number formatting
export function money(n) {
if (n == null) return "—";
if (n === 0) return "$0";
if (Math.abs(n) < 0.01) return "$" + n.toFixed(6);
return "$" + n.toFixed(4);
}
export function ms(n) {
return n == null ? "—" : `${Math.round(Number(n))} ms`;
}
export function num(n) {
if (n == null) return "—";
return Number(n).toLocaleString();
}
export function pct(n) {
if (n == null) return "—";
return (n * 100).toFixed(1) + "%";
}
export function timeAgo(unixSecs) {
const d = new Date(unixSecs * 1000);
return d.toLocaleTimeString([], { hour: "2-digit", minute: "2-digit", second: "2-digit" });
}
web/js/views/dashboard.js
Vendored
+218
View File
@@ -0,0 +1,218 @@
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
import { h, mount, money, num, ms, pct, timeAgo } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { lineChart, barList } from "../charts.js";
const RANGES = ["1h", "24h", "7d"];
const GROUPS = ["model", "tier", "label", "account"];
function kpi(big, lbl) {
return h("div", { class: "kpi" }, h("div", { class: "big" }, big), h("div", { class: "lbl" }, lbl));
}
function seg(options, current, onPick, labelFn) {
return h(
"div",
{ class: "seg" },
...options.map((o) =>
h("button", { class: o === current ? "active" : "", onClick: () => onPick(o) }, labelFn ? labelFn(o) : o)
)
);
}
export async function renderDashboard() {
let range = "24h";
let groupby = "model";
const root = h("div");
async function load() {
const rangeSecs = range;
let stats, recent, health;
try {
[stats, recent, health] = await Promise.all([
api.stats(rangeSecs, groupby),
api.requests(40, 0),
api.health().catch(() => null),
]);
} catch (e) {
mount(root, h("div", { class: "callout" }, String(e.message || e)));
return;
}
const tot = stats.totals || {};
const series = stats.series || [];
const breakdown = (stats.breakdown || []).map((b) => ({ key: b.key || t("common.none"), value: b.requests }));
mount(
root,
h(
"div",
{ class: "page-head" },
h(
"div",
{ class: "flex" },
h("div", { class: "grow" }, h("h2", {}, t("dash.title")), h("p", {}, t("dash.subtitle"))),
seg(RANGES, range, (r) => {
range = r;
load();
}, (r) => t("dash.range." + r))
)
),
// KPIs
h(
"div",
{ class: "kpis" },
kpi(num(tot.requests), t("dash.kpi.requests")),
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
kpi(money(tot.cost_usd), t("dash.kpi.cost")),
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
kpi(pct(tot.success_rate), t("dash.kpi.success")),
kpi(num(tot.failovers), t("dash.kpi.failovers"))
),
h(
"div",
{ class: "grid cols-2" },
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("dash.series.title"))),
series.length < 2 ? h("div", { class: "empty" }, t("dash.series.empty")) : lineChart(series, "requests")
),
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("dash.series.cost"))),
series.length < 2 ? h("div", { class: "empty" }, t("dash.series.empty")) : lineChart(series, "cost_usd")
)
),
// breakdown
h(
"div",
{ class: "card" },
h(
"div",
{ class: "card-head" },
h("h3", {}, t("dash.breakdown.title")),
h(
"div",
{ class: "right" },
seg(GROUPS, groupby, (g) => {
groupby = g;
load();
}, (g) => t("dash.groupby." + g))
)
),
barList(breakdown, num)
),
// recent
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("dash.recent.title"))),
recentTable(recent.requests || [])
),
health ? healthCard(health) : null
);
}
await load();
return root;
}
function recentTable(rows) {
if (!rows.length) return h("div", { class: "empty" }, t("dash.recent.empty"));
return h(
"div",
{ class: "table-wrap" },
h(
"table",
{},
h(
"thead",
{},
h(
"tr",
{},
h("th", {}, t("dash.col.time")),
h("th", {}, t("dash.col.model")),
h("th", {}, t("dash.col.tier")),
h("th", {}, t("dash.col.task")),
h("th", {}, t("dash.col.tokens")),
h("th", {}, t("dash.col.cost")),
h("th", {}, t("dash.col.latency")),
h("th", {}, t("dash.col.outcome"))
)
),
h(
"tbody",
{},
...rows.map((r) =>
h(
"tr",
{},
h("td", { class: "mono" }, timeAgo(r.ts)),
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
h("td", {}, r.task_label || "—"),
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
h("td", { class: "mono" }, money(r.cost_usd)),
h("td", { class: "mono" }, ms(r.latency_ms)),
h("td", {}, h("span", { class: "badge " + (r.outcome === "ok" ? "ok" : "error") }, r.outcome))
)
)
)
)
);
}
function healthCard(hh) {
return h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("dash.health.title"))),
h(
"div",
{ class: "flex wrap", style: "margin-bottom:12px" },
h(
"span",
{ class: "badge " + (hh.router?.reachable ? "ok" : "bad"), title: hh.router?.url || "" },
t("status.router") + ": " + (hh.router?.reachable ? t("dash.health.reachable") : t("dash.health.unreachable"))
)
),
h(
"div",
{ class: "table-wrap" },
h(
"table",
{},
h("thead", {}, h("tr", {}, h("th", {}, t("models.col.id")), h("th", {}, t("models.col.provider")), h("th", {}, t("models.col.model")), h("th", {}, t("models.col.key")))),
h(
"tbody",
{},
...(hh.models || []).map((m) =>
h(
"tr",
{},
h("td", { class: "mono" }, m.id),
h("td", {}, m.provider),
h("td", { class: "mono" }, m.model),
h("td", {}, h("span", { class: "badge " + keyBadge(m.key_source) }, m.key_source))
)
)
)
)
)
);
}
function keyBadge(src) {
if (src === "store") return "store";
if (src === "env") return "env";
if (src === "missing") return "bad";
return "";
}
web/js/views/keys.js
Vendored
+128
View File
@@ -0,0 +1,128 @@
// API keys — list, create (with copy-once), revoke.
import { h, mount, modal, toast, confirmDialog } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
export async function renderKeys() {
const modelsData = await api.listModels();
const ids = (modelsData.models || []).map((m) => m.id);
const root = h("div");
function openCreate(reload) {
const accountIn = h("input", { placeholder: "team-a" });
const rateIn = h("input", { type: "number", value: 600 });
const customIn = h("input", { placeholder: "sk-gw-…" });
const allowInputs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox" }) }));
const body = h(
"div",
{},
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
h(
"label",
{ class: "field" },
h("span", {}, t("keys.form.allow")),
h("div", { class: "flex wrap" }, ...allowInputs.map(({ id, cb }) => h("label", { class: "check" }, cb, id))),
h("div", { class: "hint" }, t("keys.form.allowHint"))
),
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
);
modal(t("keys.form.title"), body, async () => {
try {
const r = await api.createKey({
key: customIn.value.trim(),
account: accountIn.value.trim() || "default",
rate_per_min: parseInt(rateIn.value) || 0,
allow_models: allowInputs.filter(({ cb }) => cb.checked).map(({ id }) => id),
});
reload();
showCreated(r.key);
} catch (e) {
toast(String(e.message || e), "bad");
return false;
}
}, t("common.add"));
}
function showCreated(key) {
const input = h("input", { value: key, readonly: true, class: "mono" });
const copyBtn = h("button", { class: "btn", onClick: () => { input.select(); navigator.clipboard?.writeText(key); toast(t("common.copied"), "good"); } }, t("common.copy"));
modal(t("common.ok"), h("div", {}, h("p", {}, t("keys.created")), h("div", { class: "flex" }, input, copyBtn)), null);
}
async function reload() {
const data = await api.listKeys();
const keys = data.keys || [];
mount(
root,
h(
"div",
{ class: "page-head" },
h(
"div",
{ class: "flex" },
h("div", { class: "grow" }, h("h2", {}, t("keys.title")), h("p", {}, t("keys.subtitle"))),
h("button", { class: "btn primary", onClick: () => openCreate(reload) }, "+ " + t("keys.add"))
)
),
data.open_mode ? h("div", { class: "callout info" }, t("keys.openMode")) : null,
keys.length === 0
? h("div", { class: "card" }, h("div", { class: "empty" }, t("keys.empty")))
: h(
"div",
{ class: "card" },
h(
"div",
{ class: "table-wrap" },
h(
"table",
{},
h(
"thead",
{},
h("tr", {}, h("th", {}, t("keys.col.key")), h("th", {}, t("keys.col.account")), h("th", {}, t("keys.col.rate")), h("th", {}, t("keys.col.models")), h("th", { class: "right-align" }, t("common.actions")))
),
h(
"tbody",
{},
...keys.map((k) =>
h(
"tr",
{},
h("td", { class: "mono" }, k.key_masked),
h("td", {}, k.account),
h("td", { class: "mono" }, k.rate_per_min || "—"),
h("td", {}, (k.allow_models && k.allow_models.length) ? k.allow_models.join(", ") : h("span", { class: "muted" }, t("keys.allowAll"))),
h(
"td",
{ class: "right-align" },
h(
"button",
{
class: "btn sm danger",
onClick: async () => {
if (!(await confirmDialog(t("keys.revokeConfirm")))) return;
try {
await api.deleteKey(k.key);
toast(t("toast.deleted"), "good");
reload();
} catch (e) {
toast(String(e.message || e), "bad");
}
},
},
t("keys.revoke")
)
)
)
)
)
)
)
)
);
}
await reload();
return root;
}
web/js/views/login.js
Vendored
+40
View File
@@ -0,0 +1,40 @@
// Login screen — admin token entry. Shown when there is no token or on 401.
import { h } from "../ui.js";
import { t } from "../i18n.js";
import { setToken } from "../api.js";
export function renderLogin(onOk) {
const input = h("input", {
type: "password",
placeholder: t("login.tokenPlaceholder"),
autofocus: true,
});
const submit = () => {
const val = input.value.trim();
if (!val) return;
setToken(val);
onOk();
};
input.addEventListener("keydown", (e) => {
if (e.key === "Enter") submit();
});
return h(
"div",
{ class: "login-wrap" },
h(
"div",
{ class: "login-card" },
h(
"div",
{ class: "brand" },
h("b", { style: "font-size:20px" }, "allaigate"),
h("span", { class: "sub" }, t("brand.sub"))
),
h("h2", {}, t("login.title")),
h("p", { class: "sub" }, t("login.subtitle")),
h("label", { class: "field" }, h("span", {}, t("login.token")), input),
h("button", { class: "btn primary", style: "width:100%", onClick: submit }, t("login.submit")),
h("p", { class: "hint", style: "margin-top:16px" }, t("login.hint"))
)
);
}
web/js/views/models.js
Vendored
+206
View File
@@ -0,0 +1,206 @@
// Models — pool table with add/edit modal, probe, and per-model secret entry.
import { h, mount, modal, toast, confirmDialog, money } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
function field(label, control, hint) {
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
}
function opt(v, label, sel) {
return h("option", { value: v, selected: sel ? true : null }, label || v);
}
function keyBadge(src) {
const map = { store: "store", env: "env", missing: "bad", none: "" };
return h("span", { class: "badge " + (map[src] ?? "") }, src);
}
function modelForm(meta, existing) {
const m = existing || { provider: "openai", kind: "chat", cost_tier: "cheap", caps: [], price_in: 0, price_out: 0 };
const idIn = h("input", { value: m.id || "", placeholder: "local-qwen", disabled: existing ? true : null });
const providerSel = h("select", {}, ...(meta.providers || ["openai"]).map((p) => opt(p, p, p === m.provider)));
const baseUrlIn = h("input", { value: m.base_url || "", placeholder: "http://localhost:8000/v1" });
const modelIn = h("input", { value: m.model || "", placeholder: "qwen2.5-7b-instruct" });
const kindSel = h("select", {}, ...(meta.kinds || ["chat"]).map((k) => opt(k, k, k === m.kind)));
const tierSel = h("select", {}, ...(meta.cost_tiers || ["cheap"]).map((c) => opt(c, c, c === m.cost_tier)));
const priceInIn = h("input", { type: "number", step: "0.01", value: m.price_in ?? 0 });
const priceOutIn = h("input", { type: "number", step: "0.01", value: m.price_out ?? 0 });
const keyEnvIn = h("input", { value: m.api_key_env || "", placeholder: "OPENAI_API_KEY" });
const secretIn = h("input", { type: "password", placeholder: t("models.form.secretPlaceholder") });
const capInputs = (meta.caps || ["tools", "vision"]).map((c) => {
const cb = h("input", { type: "checkbox", checked: (m.caps || []).includes(c) ? true : null });
return { c, cb };
});
const node = h(
"div",
{},
h("div", { class: "row" }, field(t("models.form.id"), idIn, existing ? null : t("models.form.idHint")), field(t("models.form.provider"), providerSel)),
field(t("models.form.baseUrl"), baseUrlIn),
field(t("models.form.model"), modelIn),
h("div", { class: "row" }, field(t("models.form.kind"), kindSel), field(t("models.form.costTier"), tierSel)),
h("div", { class: "row" }, field(t("models.form.priceIn"), priceInIn), field(t("models.form.priceOut"), priceOutIn)),
field(t("models.form.caps"), h("div", { class: "flex wrap" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check" }, cb, c)))),
field(t("models.form.apiKeyEnv"), keyEnvIn, t("models.form.apiKeyEnvHint")),
field(t("models.form.secret"), secretIn)
);
const getValue = () => ({
model: {
id: existing ? m.id : idIn.value.trim(),
provider: providerSel.value,
base_url: baseUrlIn.value.trim(),
model: modelIn.value.trim(),
kind: kindSel.value,
cost_tier: tierSel.value,
price_in: parseFloat(priceInIn.value) || 0,
price_out: parseFloat(priceOutIn.value) || 0,
api_key_env: keyEnvIn.value.trim() || null,
caps: capInputs.filter(({ cb }) => cb.checked).map(({ c }) => c),
},
secret: secretIn.value,
keyEnv: keyEnvIn.value.trim(),
});
return { node, getValue };
}
async function openModal(meta, existing, reload) {
const { node, getValue } = modelForm(meta, existing);
modal(existing ? t("models.form.editTitle") : t("models.form.addTitle"), node, async () => {
const { model, secret, keyEnv } = getValue();
if (!model.id) {
toast(t("models.form.id") + "?", "bad");
return false;
}
// strip null api_key_env so serde keeps it optional
if (!model.api_key_env) delete model.api_key_env;
try {
if (existing) await api.updateModel(existing.id, model);
else await api.createModel(model);
if (secret && keyEnv) await api.setSecret(keyEnv, secret);
toast(t("toast.saved"), "good");
reload();
} catch (e) {
toast(String(e.message || e), "bad");
return false;
}
});
}
export async function renderModels() {
const meta = appState.meta || (await api.meta());
const root = h("div");
async function reload() {
const data = await api.listModels();
const models = data.models || [];
mount(
root,
h(
"div",
{ class: "page-head" },
h(
"div",
{ class: "flex" },
h("div", { class: "grow" }, h("h2", {}, t("models.title")), h("p", {}, t("models.subtitle"))),
h("button", { class: "btn primary", onClick: () => openModal(meta, null, reload) }, "+ " + t("models.add"))
)
),
models.length === 0
? h("div", { class: "card" }, h("div", { class: "empty" }, t("models.empty")))
: h(
"div",
{ class: "card" },
h(
"div",
{ class: "table-wrap" },
h(
"table",
{},
h(
"thead",
{},
h(
"tr",
{},
h("th", {}, t("models.col.id")),
h("th", {}, t("models.col.provider")),
h("th", {}, t("models.col.model")),
h("th", {}, t("models.col.tier")),
h("th", {}, t("models.col.key")),
h("th", {}, t("models.col.caps")),
h("th", {}, t("models.col.price")),
h("th", { class: "right-align" }, t("common.actions"))
)
),
h(
"tbody",
{},
...models.map((m) => row(m, meta, reload))
)
)
)
)
);
}
await reload();
return root;
}
function row(m, meta, reload) {
const probeBtn = h("button", { class: "btn sm ghost" }, t("models.probe"));
probeBtn.addEventListener("click", async () => {
probeBtn.disabled = true;
probeBtn.textContent = "…";
try {
const r = await api.probeModel(m.id);
if (r.ok) toast(t("models.probe.ok", { ms: r.latency_ms }), "good");
else toast(t("models.probe.fail", { err: r.error || "" }), "bad");
} catch (e) {
toast(String(e.message || e), "bad");
}
probeBtn.disabled = false;
probeBtn.textContent = t("models.probe");
});
return h(
"tr",
{},
h("td", { class: "mono" }, m.id),
h("td", {}, m.provider),
h("td", { class: "mono" }, m.model),
h("td", {}, h("span", { class: "badge " + (m.cost_tier || "") }, m.cost_tier || "—")),
h("td", {}, keyBadge(m.key_source || (m.api_key_env ? "missing" : "none"))),
h("td", {}, (m.caps || []).length ? h("div", { class: "flex wrap" }, ...m.caps.map((c) => h("span", { class: "badge" }, c))) : "—"),
h("td", { class: "mono" }, `${m.price_in || 0}/${m.price_out || 0}`),
h(
"td",
{ class: "right-align" },
h(
"div",
{ class: "flex", style: "justify-content:flex-end" },
probeBtn,
h("button", { class: "btn sm ghost", onClick: () => openModal(meta, m, reload) }, t("common.edit")),
h(
"button",
{
class: "btn sm danger",
onClick: async () => {
if (!(await confirmDialog(t("models.deleteConfirm", { id: m.id })))) return;
try {
await api.deleteModel(m.id);
toast(t("toast.deleted"), "good");
reload();
} catch (e) {
toast(String(e.message || e), "bad");
}
},
},
t("common.delete")
)
)
)
);
}
web/js/views/playground.js
Vendored
+137
View File
@@ -0,0 +1,137 @@
// Playground — send a prompt through the live pipeline, inspect routing decision.
import { h, mount, money, ms, num } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
export async function renderPlayground() {
const meta = appState.meta || (await api.meta());
const modelsData = await api.listModels();
const ids = (modelsData.models || []).map((m) => m.id);
let mode = "auto";
const root = h("div");
const sysIn = h("textarea", { placeholder: t("pg.systemPlaceholder"), rows: 2 });
const promptIn = h("textarea", { placeholder: t("pg.promptPlaceholder"), rows: 5 });
const tempIn = h("input", { type: "number", step: "0.1", value: 0.7 });
const maxTokIn = h("input", { type: "number", value: 512 });
const profileSel = h("select", {}, ...(meta.profiles || []).map((p) => h("option", { value: p }, p)));
const modelSel = h("select", {}, ...ids.map((id) => h("option", { value: id }, id)));
const modeSeg = h(
"div",
{ class: "seg" },
h("button", { class: "active", onClick: (e) => setMode("auto", e) }, t("pg.mode.auto")),
h("button", { onClick: (e) => setMode("pinned", e) }, t("pg.mode.pinned"))
);
function buildModeExtra() {
return h(
"label",
{ class: "field" },
h("span", {}, mode === "auto" ? t("pg.profile") : t("pg.model")),
mode === "auto" ? profileSel : modelSel
);
}
let modeExtra = buildModeExtra();
function setMode(m, e) {
mode = m;
modeSeg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
e.target.classList.add("active");
const next = buildModeExtra();
modeExtra.replaceWith(next);
modeExtra = next;
}
const out = h("div", { class: "answer" }, h("div", { class: "empty" }, t("pg.empty")));
const decision = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
const sendBtn = h("button", { class: "btn primary" }, t("pg.send"));
sendBtn.addEventListener("click", async () => {
const prompt = promptIn.value.trim();
if (!prompt) return;
const messages = [];
if (sysIn.value.trim()) messages.push({ role: "system", content: sysIn.value.trim() });
messages.push({ role: "user", content: prompt });
let model = "cortiq-auto";
if (mode === "auto") {
if (profileSel.value) model = "cortiq-auto:" + profileSel.value;
} else {
model = modelSel.value;
}
sendBtn.disabled = true;
sendBtn.textContent = t("pg.sending");
mount(out, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.sending")));
try {
const r = await api.test({
model,
messages,
temperature: parseFloat(tempIn.value),
max_tokens: parseInt(maxTokIn.value) || null,
});
if (r.ok) {
mount(out, r.answer || h("span", { class: "muted" }, "—"));
renderDecision(r);
} else {
mount(out, h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
}
} catch (e) {
mount(out, h("div", { class: "callout" }, String(e.message || e)));
}
sendBtn.disabled = false;
sendBtn.textContent = t("pg.send");
});
function kv(k, v) {
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
}
function renderDecision(r) {
const c = r.cortiq || {};
const score = c.complexity?.score ?? 0;
mount(
decision,
h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))),
kv(t("pg.task"), c.task_label || "—"),
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, c.complexity.tier) : "—"),
h(
"div",
{ class: "kv", style: "display:block" },
h("div", { class: "flex" }, h("span", { class: "k grow" }, t("pg.score")), h("span", { class: "v" }, score.toFixed(2))),
h("div", { class: "scorebar" }, h("i", { style: `width:${Math.min(100, score * 100)}%` }))
),
kv(t("pg.selected"), h("span", { class: "mono" }, c.selected_model || "—")),
kv(t("pg.source"), c.route_source || "—"),
kv(t("pg.tokens"), `${r.usage?.prompt_tokens ?? 0}/${r.usage?.completion_tokens ?? 0}`),
kv(t("pg.cost"), money(c.cost_usd)),
kv(t("pg.latency"), ms(r.latency_ms)),
kv(t("pg.failover"), c.failover ? "yes" : "no")
);
}
mount(
root,
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))),
h(
"div",
{ class: "pg" },
h(
"div",
{},
h(
"div",
{ class: "card" },
h("label", { class: "field" }, h("span", {}, t("pg.system")), sysIn),
h("label", { class: "field" }, h("span", {}, t("pg.prompt")), promptIn),
h("label", { class: "field" }, h("span", {}, t("pg.mode")), modeSeg),
modeExtra,
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("pg.temp")), tempIn), h("label", { class: "field" }, h("span", {}, t("pg.maxTokens")), maxTokIn)),
h("div", { class: "flex", style: "justify-content:flex-end" }, sendBtn)
),
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
),
decision
)
);
return root;
}
web/js/views/protocols.js
Vendored
+67
View File
@@ -0,0 +1,67 @@
// Protocols — live toggles for inbound API surfaces.
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
const ORDER = [
"openai_chat",
"openai_completions",
"openai_embeddings",
"openai_models",
"anthropic_messages",
"mcp",
"native_passthrough",
];
export async function renderProtocols() {
const meta = appState.meta || (await api.meta());
const impl = meta.protocols_impl || {};
const state = await api.getProtocols();
const root = h("div");
async function save() {
try {
await api.putProtocols(state);
toast(t("toast.saved"), "good");
} catch (e) {
toast(String(e.message || e), "bad");
}
}
function row(key) {
const implemented = !!impl[key];
const input = h("input", {
type: "checkbox",
checked: state[key] ? true : null,
disabled: implemented ? null : true,
onChange: (e) => {
state[key] = e.target.checked;
save();
},
});
return h(
"div",
{ class: "togglerow" },
h(
"div",
{ class: "meta" },
h(
"div",
{ class: "name" },
t("protocols." + key),
h("span", { class: "badge " + (implemented ? "ok" : "") }, implemented ? t("protocols.implemented") : t("protocols.planned"))
),
h("div", { class: "desc" }, t("protocols." + key + ".d"))
),
h("label", { class: "switch" }, input, h("span", { class: "slider" }))
);
}
mount(
root,
h("div", { class: "page-head" }, h("h2", {}, t("protocols.title")), h("p", {}, t("protocols.subtitle"))),
h("div", { class: "card" }, ...ORDER.map(row))
);
return root;
}
web/js/views/routing.js
Vendored
+158
View File
@@ -0,0 +1,158 @@
// Routing — tier editor (ordered model chips), default model, and policy.
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
export async function renderRouting() {
const meta = appState.meta || (await api.meta());
const [routing, modelsData] = await Promise.all([api.getRouting(), api.listModels()]);
const ids = (modelsData.models || []).map((m) => m.id);
// local editable state — ensure standard tiers exist for a friendly editor
const tiers = Object.assign({ low: [], medium: [], high: [] }, routing.tiers || {});
const st = {
tiers,
default: routing.default || ids[0] || "",
policy: routing.policy || { mode: "fixed_table", min_class: {} },
};
const root = h("div");
function moveChip(tier, i, dir) {
const arr = st.tiers[tier];
const ni = i + dir;
if (ni < 0 || ni >= arr.length) return;
[arr[i], arr[ni]] = [arr[ni], arr[i]];
render();
}
function removeChip(tier, i) {
st.tiers[tier].splice(i, 1);
render();
}
function addChip(tier, id) {
if (id && !st.tiers[tier].includes(id)) st.tiers[tier].push(id);
render();
}
function tierRow(tier) {
const arr = st.tiers[tier];
const avail = ids.filter((id) => !arr.includes(id));
const addSel = h(
"select",
{
style: "width:auto",
onChange: (e) => {
addChip(tier, e.target.value);
},
},
h("option", { value: "" }, t("routing.addModel")),
...avail.map((id) => h("option", { value: id }, id))
);
return h(
"div",
{ class: "tierrow" },
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)),
h(
"div",
{ class: "chips" },
...arr.map((id, i) =>
h(
"div",
{ class: "chip" },
h("span", { class: "ord" }, i + 1),
id,
h("span", { class: "arrow", title: "←", onClick: () => moveChip(tier, i, -1) }, "◄"),
h("span", { class: "arrow", title: "→", onClick: () => moveChip(tier, i, 1) }, "►"),
h("span", { class: "x", onClick: () => removeChip(tier, i) }, "×")
)
),
avail.length ? addSel : null
)
);
}
function policyCard() {
const modeSel = h(
"select",
{ onChange: (e) => { st.policy.mode = e.target.value; render(); } },
h("option", { value: "fixed_table", selected: st.policy.mode === "fixed_table" }, t("routing.policy.fixed")),
h("option", { value: "cost_aware", selected: st.policy.mode === "cost_aware" }, t("routing.policy.costAware"))
);
const children = [
h("label", { class: "field" }, h("span", {}, t("routing.policy.mode")), modeSel),
];
if (st.policy.mode === "cost_aware") {
const maxIn = h("input", {
type: "number",
step: "0.01",
value: st.policy.max_cost_usd_per_request ?? "",
placeholder: "0.50",
onChange: (e) => { st.policy.max_cost_usd_per_request = e.target.value === "" ? null : parseFloat(e.target.value); },
});
children.push(h("label", { class: "field" }, h("span", {}, t("routing.policy.maxCost")), maxIn));
st.policy.min_class = st.policy.min_class || {};
const rows = Object.keys(st.tiers).map((tier) => {
const sel = h(
"select",
{ onChange: (e) => { st.policy.min_class[tier] = e.target.value; } },
h("option", { value: "" }, "—"),
...(meta.cost_tiers || []).map((c) => h("option", { value: c, selected: st.policy.min_class[tier] === c }, c))
);
return h("label", { class: "field" }, h("span", {}, t("routing.policy.minClass") + " · " + tier), sel);
});
children.push(h("div", { class: "row" }, ...rows));
}
return h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("routing.policy"))), ...children);
}
function render() {
const defSel = h(
"select",
{ onChange: (e) => { st.default = e.target.value; } },
...ids.map((id) => h("option", { value: id, selected: id === st.default }, id))
);
const saveBtn = h("button", { class: "btn primary" }, t("common.save"));
saveBtn.addEventListener("click", async () => {
saveBtn.disabled = true;
try {
await api.putRouting({ tiers: st.tiers, default: st.default, policy: st.policy });
toast(t("toast.saved"), "good");
} catch (e) {
toast(String(e.message || e), "bad");
}
saveBtn.disabled = false;
});
mount(
root,
h(
"div",
{ class: "page-head" },
h(
"div",
{ class: "flex" },
h("div", { class: "grow" }, h("h2", {}, t("routing.title")), h("p", {}, t("routing.subtitle"))),
saveBtn
)
),
ids.length === 0 ? h("div", { class: "callout" }, t("routing.empty")) : null,
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))),
...Object.keys(st.tiers).map(tierRow)
),
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("routing.default"))),
h("label", { class: "field" }, h("span", {}, t("routing.defaultHint")), defSel)
),
policyCard()
);
}
render();
return root;
}
web/js/views/settings.js
Vendored
+191
View File
@@ -0,0 +1,191 @@
// Settings — router/route/breaker/log/telemetry/stats + export/import config.
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
function field(label, control, hint) {
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
}
function check(label, checked) {
const cb = h("input", { type: "checkbox", checked: checked ? true : null });
return { node: h("label", { class: "check" }, cb, label), cb };
}
export async function renderSettings() {
const meta = appState.meta || (await api.meta());
const s = await api.getSettings();
const root = h("div");
const listenIn = h("input", { value: s.listen || "" });
// router
const rUrl = h("input", { value: s.router?.url || "" });
const rKeyEnv = h("input", { value: s.router?.api_key_env || "", placeholder: "CORTIQ_ROUTER_KEY" });
const rTimeout = h("input", { type: "number", value: s.router?.timeout_ms ?? 800 });
const rTax = h("input", { value: s.router?.taxonomy_id || "" });
const rVerify = check(t("settings.router.verifyTls"), s.router?.verify_tls);
// route
const strategySel = h("select", {}, ...(meta.text_strategies || []).map((x) => h("option", { value: x, selected: x === s.route?.text_strategy }, x)));
const maxChars = h("input", { type: "number", value: s.route?.max_chars ?? 4000 });
const cacheTtl = h("input", { value: s.route?.cache_ttl || "60s" });
const profileSel = h("select", {}, ...(meta.profiles || []).map((x) => h("option", { value: x, selected: x === s.route?.profile }, x)));
// breaker
const brThresh = h("input", { type: "number", value: s.breaker?.threshold ?? 5 });
const brCool = h("input", { value: s.breaker?.cooldown || "30s" });
// log
const logSel = h("select", {}, ...["error", "warn", "info", "debug", "trace"].map((x) => h("option", { value: x, selected: x === s.log?.level }, x)));
const logBodies = check(t("settings.log.bodies"), s.log?.bodies);
// telemetry / cortiq
const metricsCb = check(t("settings.telemetry.metrics"), s.telemetry?.metrics);
const echoCb = check(t("settings.cortiq.echo"), s.cortiq?.echo);
// stats
const statsEnabled = check(t("settings.stats.enabled"), s.stats?.enabled);
const statsFile = h("input", { value: s.stats?.file || "" });
const statsRet = h("input", { value: s.stats?.retention || "7d" });
const saveBtn = h("button", { class: "btn primary" }, t("common.save"));
saveBtn.addEventListener("click", async () => {
saveBtn.disabled = true;
const patch = {
listen: listenIn.value.trim(),
router: {
url: rUrl.value.trim(),
api_key_env: rKeyEnv.value.trim() || null,
timeout_ms: parseInt(rTimeout.value) || 800,
verify_tls: rVerify.cb.checked,
taxonomy_id: rTax.value.trim() || null,
},
route: {
text_strategy: strategySel.value,
max_chars: parseInt(maxChars.value) || 4000,
cache_ttl: cacheTtl.value.trim() || "60s",
profile: profileSel.value,
},
breaker: { threshold: parseInt(brThresh.value) || 5, cooldown: brCool.value.trim() || "30s" },
log: { level: logSel.value, bodies: logBodies.cb.checked },
telemetry: { metrics: metricsCb.cb.checked, otlp_endpoint_env: s.telemetry?.otlp_endpoint_env || null },
cortiq: { echo: echoCb.cb.checked },
stats: {
enabled: statsEnabled.cb.checked,
file: statsFile.value.trim(),
retention: statsRet.value.trim() || "7d",
ring_size: s.stats?.ring_size ?? 500,
},
};
// serde: drop null optionals
if (!patch.router.api_key_env) delete patch.router.api_key_env;
if (!patch.router.taxonomy_id) delete patch.router.taxonomy_id;
if (!patch.telemetry.otlp_endpoint_env) delete patch.telemetry.otlp_endpoint_env;
try {
const r = await api.putSettings(patch);
toast(r.needs_restart ? t("settings.needsRestart") : t("toast.saved"), r.needs_restart ? "warn" : "good");
} catch (e) {
toast(String(e.message || e), "bad");
}
saveBtn.disabled = false;
});
async function exportConfig() {
const cfg = await api.getConfig();
const blob = new Blob([JSON.stringify(cfg, null, 2)], { type: "application/json" });
const url = URL.createObjectURL(blob);
const a = h("a", { href: url, download: "gateway-config.json" });
document.body.appendChild(a);
a.click();
a.remove();
URL.revokeObjectURL(url);
}
const fileIn = h("input", { type: "file", accept: ".json,application/json", style: "display:none" });
fileIn.addEventListener("change", async () => {
const f = fileIn.files[0];
if (!f) return;
try {
const cfg = JSON.parse(await f.text());
await api.putConfig(cfg);
toast(t("toast.imported"), "good");
} catch (e) {
toast(String(e.message || e), "bad");
}
fileIn.value = "";
});
mount(
root,
h(
"div",
{ class: "page-head" },
h(
"div",
{ class: "flex" },
h("div", { class: "grow" }, h("h2", {}, t("settings.title")), h("p", {}, t("settings.subtitle"))),
saveBtn
)
),
h(
"div",
{ class: "grid cols-2" },
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.router"))),
field(t("settings.router.url"), rUrl),
field(t("settings.router.keyEnv"), rKeyEnv),
h("div", { class: "row" }, field(t("settings.router.timeout"), rTimeout), field(t("settings.router.taxonomy"), rTax)),
rVerify.node
),
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.route"))),
field(t("settings.route.strategy"), strategySel),
h("div", { class: "row" }, field(t("settings.route.maxChars"), maxChars), field(t("settings.route.cacheTtl"), cacheTtl)),
field(t("settings.route.profile"), profileSel)
),
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.breaker"))),
h("div", { class: "row" }, field(t("settings.breaker.threshold"), brThresh), field(t("settings.breaker.cooldown"), brCool))
),
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.log"))),
field(t("settings.log.level"), logSel),
logBodies.node
),
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.telemetry"))),
metricsCb.node,
h("div", { class: "divider" }),
h("div", { class: "card-head" }, h("h3", {}, t("settings.cortiq"))),
echoCb.node
),
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.stats"))),
statsEnabled.node,
field(t("settings.stats.file"), statsFile),
field(t("settings.stats.retention"), statsRet)
)
),
h(
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.listen"))),
field(t("settings.listen"), listenIn),
h("div", { class: "divider" }),
h(
"div",
{ class: "flex wrap" },
h("button", { class: "btn", onClick: exportConfig }, t("settings.export")),
h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")),
fileIn
)
)
);
return root;
}