release: v0.2.37 — zero-argument startup, first-run wizard, real download progress
Startup no longer needs arguments: the config is created on first launch, a busy port is swapped for a free one, the console is open by default, and the browser lands on a setup wizard that downloads a ready-made .cmf model and wires it into routing (managed server + routing.default) when you press Finish. Download progress was reported from curl's own meter, which prints a full 0→100% bar for the -L redirect hop before the transfer starts — so every job showed 100% and then hung. Progress now comes from bytes on disk against the size from the HuggingFace tree (or a HEAD request). The UI and the gateway also disagreed about the output file name, so a finished download never marked its card as downloaded; the gateway now returns the exact stem and the UI sends it back verbatim for download, progress and delete. Featured models are read live from the infosave HF account in both the wizard and Import, and all 432 UI strings now exist in every one of the 7 languages.
This commit is contained in:
@@ -36,6 +36,7 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Zero-config start.** `cortiq-gateway` with no flags: the config is created for you, a busy port is swapped for a free one, the browser opens on a setup wizard that downloads a ready-made local model and wires it into routing — no TOML, no token, no separate model server.
|
||||
- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
|
||||
- **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
|
||||
@@ -60,6 +61,7 @@ docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
|
||||
# from crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # no flags, no config file, no token — see Quick start
|
||||
|
||||
# or build from source
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -102,32 +104,54 @@ The gateway ships with an embedded web console at **`/admin`**:
|
||||
|  |
|
||||
| test the live pipeline and see the routing decision |
|
||||
|
||||
The console is **open by default** — no token needed. To protect it, set any one of
|
||||
`--admin-token <TOKEN>`, `[admin].token`, or `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <YOUR_TOKEN>
|
||||
cortiq-gateway --admin-token <YOUR_TOKEN>
|
||||
# open http://localhost:9000/admin?token=<YOUR_TOKEN>
|
||||
```
|
||||
|
||||
If `--admin-token` / `[admin].token_env` are not set, a token is generated at startup
|
||||
and printed to the log. All `/admin/api/*` endpoints require a Bearer admin token;
|
||||
secret values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
Once a token is set, every `/admin/api/*` endpoint requires it as a Bearer token. Secret
|
||||
values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick start
|
||||
|
||||
```bash
|
||||
# 1. start a router (the hosted allaigate router, or a local cortiq-router)
|
||||
# it listens e.g. on http://localhost:8080 (or https://api.allaigate.com)
|
||||
|
||||
# 2. describe your model pool
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. run the gateway
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
# Gateway listens on 0.0.0.0:9000 and serves the admin console at /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
That is the whole command. On first run the gateway:
|
||||
|
||||
1. **creates its config** — `config/gateway.toml` when started from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml` (override with `$CORTIQ_GATEWAY_HOME` or `--config`);
|
||||
2. **picks a free port** if 9000 is taken, and writes the choice back to the config;
|
||||
3. **opens your browser** on the setup wizard at `/admin#/onboarding`.
|
||||
|
||||
The wizard takes you from nothing to a working local model:
|
||||
|
||||
| Step | What happens |
|
||||
|---|---|
|
||||
| **Language** | 7 languages; changeable later from the header |
|
||||
| **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress |
|
||||
| **Any HF model** | search HuggingFace and auto-convert to `.cmf` |
|
||||
| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, and `routing.default` points at it |
|
||||
|
||||
No TOML editing, no separate "register" step, no manually started model server. Prefer to skip
|
||||
the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config —
|
||||
both still work, and changes hot-reload without a restart.
|
||||
|
||||
```bash
|
||||
# explicit config / port / no browser — all optional
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
For smart routing across a pool, point `[router]` at the hosted allaigate router or a local
|
||||
`cortiq-router` (see [docs/ROUTER.md](docs/ROUTER.md)). Without a router the gateway degrades
|
||||
gracefully to the default model, so a local-only setup works out of the box.
|
||||
|
||||
Now any OpenAI client works through the gateway:
|
||||
|
||||
```python
|
||||
|
||||
Reference in New Issue
Block a user