Compare commits

...
10 Commits
Author SHA1 Message Date
Sanaullah Turab d737946aac fix(datetime): replace deprecated utcnow() with timezone-aware equivalent (#15)
CI / Backend Tests (Python 3.10) (push) Canceled after 0s
CI / Backend Tests (Python 3.11) (push) Canceled after 0s
CI / Backend Tests (Python 3.12) (push) Canceled after 0s
CI / Frontend Build & Typecheck (push) Canceled after 0s
- Replaced datetime.utcnow() with datetime.now(timezone.utc) in router and observability modules
- Preserves backwards compatibility with Python 3.10+
- Fixes #14
2026-09-11 19:18:56 +05:30
Sanaullah Turab 077b3e2af4 feat(cli): add --json output flag to doctor, models, route, run, and analytics (#13)
- Adds optional `--json` / `-j` flag to CLI commands for scriptable output
- Redirects non-JSON logging messages to stderr under JSON mode
- Adds test suite for CLI JSON serialization in backend/tests/test_cli.py
- Updates documentation in README.md
2026-09-11 19:15:22 +05:30
Vardhman Gupta e6d9640112 fix: package standalone app.server module and bump model-router-cli to 1.0.2 2026-09-11 13:30:23 +05:30
Vardhman Gupta 2e013188b1 feat: package distribution with PyPI support, embedded UI, Dockerfile, and CLI enhancements 2026-09-11 13:14:33 +05:30
Vardhman Gupta 02e0f1803c Merge pull request #12 from lionkingchuja/feature/add-csv-json-export
Add traffic telemetry export
2026-09-11 09:01:34 +05:30
lionkingchuja 1f20c725cd Add traffic telemetry export 2026-09-11 02:57:58 +05:30
Vardhman Gupta cd6f7def83 Merge pull request #11 from Sanaullah-Turab/feat/theme-switcher 2026-09-09 22:39:03 +05:30
Sanaullah-Turab 8113da6c12 update documentation 2026-09-09 18:25:27 +05:00
Sanaullah-Turab 1a45cf5075 feat: add dark/light theme switching with ThemeContext and localStorage persistence 2026-09-09 18:12:16 +05:00
Vardhman Gupta 54e140e4ed chore: add model_router_project.md to .gitignore and untrack docs 2026-09-09 18:04:04 +05:30
20 changed files with 887 additions and 1348 deletions
+5
View File
@@ -21,11 +21,14 @@ model_router.db-journal
.pytest_cache/ .pytest_cache/
.coverage .coverage
htmlcov/ htmlcov/
build/
*.egg-info/
# Node / Frontend # Node / Frontend
node_modules/ node_modules/
dist/ dist/
.dist/ .dist/
backend/app/static/
npm-debug.log* npm-debug.log*
yarn-debug.log* yarn-debug.log*
yarn-error.log* yarn-error.log*
@@ -43,3 +46,5 @@ Thumbs.db
# Project Interview Guide & Docs # Project Interview Guide & Docs
project.md project.md
PROJECT.md PROJECT.md
model_router_project.md
docs/model_router_project.md
+33
View File
@@ -0,0 +1,33 @@
# Stage 1: Build React Frontend UI
FROM node:20-alpine AS frontend-builder
WORKDIR /app/frontend
COPY frontend/package*.json ./
RUN npm ci
COPY frontend/ ./
RUN npm run build
# Stage 2: Python Backend Runtime
FROM python:3.11-slim
WORKDIR /app
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
ENV PORT=8000
ENV HOST=0.0.0.0
# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy backend application code
COPY backend/ ./backend/
# Copy compiled static frontend into backend/app/static
COPY --from=frontend-builder /app/frontend/dist ./backend/app/static
EXPOSE 8000
CMD ["python", "backend/main.py"]
+64 -10
View File
@@ -1,5 +1,9 @@
# Model Router # Model Router
[![PyPI - Version](https://img.shields.io/pypi/v/model-router-cli?logo=pypi&logoColor=white&color=blue)](https://pypi.org/project/model-router-cli/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/model-router-cli?logo=python&logoColor=white)](https://pypi.org/project/model-router-cli/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
Intelligent, explainable, cost- and latency-aware LLM request routing platform and AI Traffic Control Room. Intelligent, explainable, cost- and latency-aware LLM request routing platform and AI Traffic Control Room.
--- ---
@@ -19,7 +23,7 @@ The platform is designed local-first, allowing full local development and testin
- **Provider Abstraction**: Decoupled adapters for Mock (simulation), Ollama (local), OpenAI, Anthropic, and Google Gemini. - **Provider Abstraction**: Decoupled adapters for Mock (simulation), Ollama (local), OpenAI, Anthropic, and Google Gemini.
- **Resilience and Tiered Fallback**: Automated retry classification for transient errors (timeouts, HTTP 429, 503) and tiered fallback to local/mock alternatives. - **Resilience and Tiered Fallback**: Automated retry classification for transient errors (timeouts, HTTP 429, 503) and tiered fallback to local/mock alternatives.
- **Budget Control Guards**: Real-time spend tracking with automated threshold interventions (80% cost optimization, 95% local-only saver, 100% block). - **Budget Control Guards**: Real-time spend tracking with automated threshold interventions (80% cost optimization, 95% local-only saver, 100% block).
- **Traffic Control Room UI**: Real-time dark operational interface featuring live topology graphs, playground inspector, SSE live request stream, visual rules builder, and cost savings simulator. - **Traffic Control Room UI**: Real-time operational interface with seamless dark/light mode switching, featuring live topology graphs, playground inspector, SSE live request stream, telemetry export as CSV/JSON, visual rules builder, and cost savings simulator.
- **Developer CLI**: Terminal diagnostics (`doctor`), routing dry-run (`route`), execution (`run`), model catalog (`models`), and analytics (`analytics`). - **Developer CLI**: Terminal diagnostics (`doctor`), routing dry-run (`route`), execution (`run`), model catalog (`models`), and analytics (`analytics`).
--- ---
@@ -66,9 +70,32 @@ The platform is designed local-first, allowing full local development and testin
- Node.js 18+ and npm - Node.js 18+ and npm
- (Optional) [Ollama](https://ollama.com/) for local model inference - (Optional) [Ollama](https://ollama.com/) for local model inference
## 📦 Quick Installation (via PyPI)
Install Model Router and its embedded **AI Traffic Control Room** dashboard directly from PyPI:
```bash
pip install model-router-cli
```
### Quick Commands:
```bash
# Run system diagnostics & environment checks
modelrouter doctor
# Launch the Traffic Control Room Dashboard & API Gateway in your browser
modelrouter ui --open-browser
# Route a prompt and inspect the explainability decision (Dry Run)
modelrouter route "Write an optimized async task worker in Python"
# Route and execute inference through the optimal provider
modelrouter run "Explain the difference between zero-shot and few-shot prompting"
```
--- ---
## Installation & Setup ## Installation & Setup (from Source)
### 1. Clone the Repository ### 1. Clone the Repository
```bash ```bash
@@ -152,23 +179,30 @@ npm run dev
## CLI Usage ## CLI Usage
The built-in Typer CLI provides terminal commands for inspection, diagnostics, and testing: The built-in Typer CLI provides terminal commands for inspection, diagnostics, and testing (use `modelrouter` if installed via pip, or `python backend/app/cli/main.py` from source).
You can append `--json` or `-j` to most commands to output raw, pipeable JSON instead of Rich formatted tables:
```bash ```bash
# Launch the AI Traffic Control Room Web UI and API Gateway
modelrouter ui --open-browser
# Run system diagnostics & provider health checks # Run system diagnostics & provider health checks
python backend/app/cli/main.py doctor modelrouter doctor
# Inspect routing decision for a prompt without executing (Dry Run) # Inspect routing decision for a prompt without executing (Dry Run)
python backend/app/cli/main.py route "Write a Python function to parse JSON" modelrouter route "Write a Python function to parse JSON"
# Route and execute a query through the selected model # Route and execute a query through the selected model
python backend/app/cli/main.py run "Debug this distributed async deadlock in worker pool" modelrouter run "Debug this distributed async deadlock in worker pool"
# List all registered models in the catalog # List all registered models in the catalog
python backend/app/cli/main.py models modelrouter models
# View recent routed traffic logs
modelrouter traffic
# View system-wide routing performance and cost savings analytics # View system-wide routing performance and cost savings analytics
python backend/app/cli/main.py analytics modelrouter analytics
``` ```
--- ---
@@ -194,16 +228,35 @@ curl -X POST http://127.0.0.1:8000/api/generate \
curl http://127.0.0.1:8000/api/models curl http://127.0.0.1:8000/api/models
``` ```
### 4. Export Historical Traffic (`GET /api/traffic/export`)
Export all persisted traffic records for auditing, accounting, or latency analysis:
```bash
# Export as JSON
curl -OJ "http://127.0.0.1:8000/api/traffic/export?format=json"
# Export as CSV
curl -OJ "http://127.0.0.1:8000/api/traffic/export?format=csv"
```
Each export includes the timestamp, request ID, prompt preview, task type, complexity, selected model, input/output/total tokens, cost saved, and total latency. The `format` query parameter accepts only `csv` or `json`.
The same export is available in the frontend under **Traffic**. Select `CSV` or `JSON` beside **Export Telemetry**, then click the button to download the complete historical traffic dataset.
--- ---
## Running Tests ## Running Tests
The test suite includes 15 automated unit, integration, and end-to-end tests covering prompt heuristics, candidate pruning, scoring weights, provider execution, error fallbacks, and REST endpoints: The test suite includes 18 automated unit, integration, and end-to-end tests covering prompt heuristics, candidate pruning, scoring weights, provider execution, error fallbacks, REST endpoints, and CSV/JSON traffic exports:
```bash ```bash
# Run backend test suite # Run the backend test suite from the repository root
pytest backend/tests pytest backend/tests
# Or run it from the backend directory
cd backend
python -m pytest tests
# Run frontend production build test # Run frontend production build test
cd frontend cd frontend
npm run build npm run build
@@ -282,6 +335,7 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
Maintained and architected by **PicadoLabs**. Maintained and architected by **PicadoLabs**.
- **Maintainer**: [@Kaap10](https://github.com/Kaap10)
- **Organization**: [PicadoLabs](https://github.com/PicadoLabs) - **Organization**: [PicadoLabs](https://github.com/PicadoLabs)
- **Website**: [https://picadolabs.me](https://picadolabs.me) - **Website**: [https://picadolabs.me](https://picadolabs.me)
- **Contact**: [picadolabs@gmail.com](mailto:picadolabs@gmail.com) - **Contact**: [picadolabs@gmail.com](mailto:picadolabs@gmail.com)
+57
View File
@@ -1,8 +1,11 @@
import uuid import uuid
import time import time
import asyncio import asyncio
import csv
import io
from typing import List, Optional, Dict, Any from typing import List, Optional, Dict, Any
from fastapi import APIRouter, Depends, HTTPException, Query from fastapi import APIRouter, Depends, HTTPException, Query
from fastapi.responses import JSONResponse, StreamingResponse
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy import select, desc from sqlalchemy import select, desc
@@ -386,6 +389,60 @@ async def get_traffic_feed(limit: int = 50, db: AsyncSession = Depends(get_db)):
return res.scalars().all() return res.scalars().all()
@router.get("/api/traffic/export")
async def export_traffic(
format: str = Query("json", pattern="^(csv|json)$"),
db: AsyncSession = Depends(get_db),
):
res = await db.execute(select(RequestRecord).order_by(desc(RequestRecord.timestamp)))
records = res.scalars().all()
fields = [
"timestamp",
"request_id",
"prompt_preview",
"task_type",
"complexity",
"selected_model",
"input_tokens",
"output_tokens",
"total_tokens",
"cost_saved",
"total_latency_ms",
]
rows = [
{
"timestamp": record.timestamp.isoformat() if record.timestamp else "",
"request_id": record.request_id,
"prompt_preview": record.prompt[:200],
"task_type": record.task_type,
"complexity": record.complexity,
"selected_model": record.selected_model,
"input_tokens": record.input_tokens,
"output_tokens": record.output_tokens,
"total_tokens": record.total_tokens,
"cost_saved": record.cost_saved,
"total_latency_ms": record.total_latency_ms,
}
for record in records
]
if format == "json":
return JSONResponse(
content=rows,
headers={"Content-Disposition": "attachment; filename=traffic-export.json"},
)
output = io.StringIO()
writer = csv.DictWriter(output, fieldnames=fields)
writer.writeheader()
writer.writerows(rows)
return StreamingResponse(
iter([output.getvalue()]),
media_type="text/csv",
headers={"Content-Disposition": "attachment; filename=traffic-export.csv"},
)
@router.get("/api/decisions/{decision_id_or_request_id}") @router.get("/api/decisions/{decision_id_or_request_id}")
async def get_decision_details(decision_id_or_request_id: str, db: AsyncSession = Depends(get_db)): async def get_decision_details(decision_id_or_request_id: str, db: AsyncSession = Depends(get_db)):
res_dec = await db.execute( res_dec = await db.execute(
+194 -90
View File
@@ -1,8 +1,10 @@
import sys import sys
import os import os
import asyncio import asyncio
import httpx import httpx
import typer import typer
import json
from typing import Any, Callable
from rich.console import Console from rich.console import Console
from rich.table import Table from rich.table import Table
from rich.panel import Panel from rich.panel import Panel
@@ -29,62 +31,90 @@ from sqlalchemy import select
app = typer.Typer(help="Model Router: Intelligent LLM Request Routing Platform") app = typer.Typer(help="Model Router: Intelligent LLM Request Routing Platform")
console = Console(safe_box=True) console = Console(safe_box=True)
err_console = Console(stderr=True, safe_box=True)
settings = get_settings() settings = get_settings()
def _render_output(json_output: bool, data: Any, render_fn: Callable[[], None]) -> None:
if json_output:
if hasattr(data, "model_dump_json"):
print(data.model_dump_json(indent=2))
elif isinstance(data, list) and all(hasattr(x, "model_dump") for x in data):
print(json.dumps([m.model_dump(mode="json") for m in data], indent=2))
else:
print(json.dumps(data, indent=2))
else:
render_fn()
@app.command() @app.command()
def doctor(): def doctor(
json_output: bool = typer.Option(False, "--json", "-j", help="Output raw JSON instead of formatted tables"),
):
"""Run environment, connectivity, and dependency health checks.""" """Run environment, connectivity, and dependency health checks."""
console.print(Panel.fit("[bold cyan]Model Router Diagnostics (modelrouter doctor)[/bold cyan]")) data = {
"python": {"version": f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}", "ok": True},
# 1. Python Check "database": {"ok": False, "message": ""},
py_ver = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}" "ollama": {"ok": False, "models": [], "message": ""},
console.print(f" [green][OK][/green] Python: [bold]{py_ver}[/bold]") "mock_engine": {"ok": True},
"providers": {
# 2. Database Check "openai": {"configured": bool(settings.OPENAI_API_KEY)},
"anthropic": {"configured": bool(settings.ANTHROPIC_API_KEY)},
"gemini": {"configured": bool(settings.GEMINI_API_KEY)},
}
}
try: try:
asyncio.run(init_db()) asyncio.run(init_db())
console.print(" [green][OK][/green] SQLite Database: [bold]Initialized & Ready[/bold]") data["database"]["ok"] = True
data["database"]["message"] = "Initialized & Ready"
except Exception as exc: except Exception as exc:
console.print(f" [red][FAIL][/red] Database Error: {exc}") data["database"]["message"] = str(exc)
# 3. Ollama Connectivity Check
ollama_ok = False
try: try:
r = httpx.get(f"{settings.OLLAMA_BASE_URL}/api/tags", timeout=1.5) r = httpx.get(f"{settings.OLLAMA_BASE_URL}/api/tags", timeout=1.5)
if r.status_code == 200: if r.status_code == 200:
models = [m.get("name") for m in r.json().get("models", [])] models = [m.get("name") for m in r.json().get("models", [])]
console.print(f" [green][OK][/green] Ollama (Local): [bold]Connected[/bold] ({len(models)} local models found)") data["ollama"]["ok"] = True
ollama_ok = True data["ollama"]["models"] = models
except Exception: except Exception:
pass pass
if not ollama_ok:
console.print(f" [yellow][!][/yellow] Ollama (Local): [dim]Not running on {settings.OLLAMA_BASE_URL} (Optional, local fallback available)[/dim]") if not data["ollama"]["ok"]:
data["ollama"]["message"] = f"Not running on {settings.OLLAMA_BASE_URL}"
# 4. Mock Engine def _render() -> None:
console.print(" [green][OK][/green] Mock Engine: [bold]Ready (Zero API keys required)[/bold]") console.print(Panel.fit("[bold cyan]Model Router Diagnostics (modelrouter doctor)[/bold cyan]"))
console.print(f" [green][OK][/green] Python: [bold]{data['python']['version']}[/bold]")
if data["database"]["ok"]:
console.print(" [green][OK][/green] SQLite Database: [bold]Initialized & Ready[/bold]")
else:
console.print(f" [red][FAIL][/red] Database Error: {data['database']['message']}")
if data["ollama"]["ok"]:
console.print(f" [green][OK][/green] Ollama (Local): [bold]Connected[/bold] ({len(data['ollama']['models'])} local models found)")
else:
console.print(f" [yellow][!][/yellow] Ollama (Local): [dim]Not running on {settings.OLLAMA_BASE_URL} (Optional, local fallback available)[/dim]")
console.print(" [green][OK][/green] Mock Engine: [bold]Ready (Zero API keys required)[/bold]")
for prov, name in [("openai", "OpenAI"), ("anthropic", "Anthropic"), ("gemini", "Gemini")]:
if data["providers"][prov]["configured"]:
console.print(f" [green][OK][/green] {name} Provider: [bold]API Key Configured in .env[/bold]")
else:
console.print(f" [yellow][!][/yellow] {name} Provider: [dim]Not configured (Optional)[/dim]")
console.print("\n[bold green]Ready for local-first intelligent routing![/bold green]\n")
# 5. External Providers _render_output(json_output, data, _render)
if settings.OPENAI_API_KEY:
console.print(" [green][OK][/green] OpenAI Provider: [bold]API Key Configured in .env[/bold]")
else:
console.print(" [yellow][!][/yellow] OpenAI Provider: [dim]Not configured (Optional)[/dim]")
if settings.ANTHROPIC_API_KEY:
console.print(" [green][OK][/green] Anthropic Provider: [bold]API Key Configured in .env[/bold]")
else:
console.print(" [yellow][!][/yellow] Anthropic Provider: [dim]Not configured (Optional)[/dim]")
if settings.GEMINI_API_KEY:
console.print(" [green][OK][/green] Gemini Provider: [bold]API Key Configured in .env[/bold]")
else:
console.print(" [yellow][!][/yellow] Gemini Provider: [dim]Not configured (Optional)[/dim]")
console.print("\n[bold green]Ready for local-first intelligent routing![/bold green]\n")
@app.command() @app.command()
def models(): def models(
json_output: bool = typer.Option(False, "--json", "-j", help="Output raw JSON instead of formatted tables"),
):
"""List all registered candidate models and their capability scores.""" """List all registered candidate models and their capability scores."""
async def _list(): async def _list():
await init_db() await init_db()
@@ -92,27 +122,31 @@ def models():
res = await db.execute(select(ModelRecord).order_by(ModelRecord.tier, ModelRecord.name)) res = await db.execute(select(ModelRecord).order_by(ModelRecord.tier, ModelRecord.name))
records = res.scalars().all() records = res.scalars().all()
table = Table(title="Model Router Registry", border_style="bright_blue") def _render() -> None:
table.add_column("Model ID", style="cyan", no_wrap=True) table = Table(title="Model Router Registry", border_style="bright_blue")
table.add_column("Tier", style="magenta") table.add_column("Model ID", style="cyan", no_wrap=True)
table.add_column("Provider", style="green") table.add_column("Tier", style="magenta")
table.add_column("Context", style="yellow") table.add_column("Provider", style="green")
table.add_column("Quality", style="blue") table.add_column("Context", style="yellow")
table.add_column("Speed", style="cyan") table.add_column("Quality", style="blue")
table.add_column("Cost / 1K In/Out", style="white") table.add_column("Speed", style="cyan")
table.add_column("Cost / 1K In/Out", style="white")
for m in records: for m in records:
cost_str = f"${m.cost_per_input_token*1000:.4f} / ${m.cost_per_output_token*1000:.4f}" if m.cost_per_input_token > 0 else "Free ($0)" cost_str = f"${m.cost_per_input_token*1000:.4f} / ${m.cost_per_output_token*1000:.4f}" if m.cost_per_input_token > 0 else "Free ($0)"
table.add_row( table.add_row(
m.id, m.id,
m.tier, m.tier,
m.provider, m.provider,
f"{m.context_window:,}", f"{m.context_window:,}",
f"{m.quality_score*100:.0f}%", f"{m.quality_score*100:.0f}%",
f"{m.speed_score*100:.0f}%", f"{m.speed_score*100:.0f}%",
cost_str, cost_str,
) )
console.print(table) console.print(table)
data = [db_model_to_meta(m) for m in records]
_render_output(json_output, data, _render)
asyncio.run(_list()) asyncio.run(_list())
@@ -121,6 +155,7 @@ def models():
def route( def route(
prompt: str = typer.Argument(..., help="Prompt text to route"), prompt: str = typer.Argument(..., help="Prompt text to route"),
policy: str = typer.Option("balanced", "--policy", "-p", help="Routing policy to use"), policy: str = typer.Option("balanced", "--policy", "-p", help="Routing policy to use"),
json_output: bool = typer.Option(False, "--json", "-j", help="Output raw JSON instead of formatted tables"),
): ):
"""Analyze prompt and explain why the optimal model was selected (Dry Run).""" """Analyze prompt and explain why the optimal model was selected (Dry Run)."""
async def _route(): async def _route():
@@ -148,24 +183,27 @@ def route(
policy_name=policy, policy_name=policy,
) )
console.print(Panel.fit( def _render() -> None:
f"[bold white]Prompt:[/bold white] \"{prompt}\"\n" console.print(Panel.fit(
f"[bold cyan]Task Type:[/bold cyan] {analysis.task_type.value} | [bold cyan]Complexity:[/bold cyan] {analysis.complexity_label.value} ({analysis.complexity:.2f})\n" f"[bold white]Prompt:[/bold white] \"{prompt}\"\n"
f"[bold green]Selected Model:[/bold green] [bold yellow]{decision.selected_model_name}[/bold yellow] ({decision.selected_model})\n" f"[bold cyan]Task Type:[/bold cyan] {analysis.task_type.value} | [bold cyan]Complexity:[/bold cyan] {analysis.complexity_label.value} ({analysis.complexity:.2f})\n"
f"[bold magenta]Confidence:[/bold magenta] {decision.confidence * 100:.0f}%\n" f"[bold green]Selected Model:[/bold green] [bold yellow]{decision.selected_model_name}[/bold yellow] ({decision.selected_model})\n"
f"[bold blue]Estimated Latency:[/bold blue] {decision.estimated_latency_ms:.0f}ms | [bold blue]Estimated Cost:[/bold blue] ${decision.estimated_cost_usd:.6f}", f"[bold magenta]Confidence:[/bold magenta] {decision.confidence * 100:.0f}%\n"
title="ROUTING DECISION", f"[bold blue]Estimated Latency:[/bold blue] {decision.estimated_latency_ms:.0f}ms | [bold blue]Estimated Cost:[/bold blue] ${decision.estimated_cost_usd:.6f}",
border_style="bright_blue", title="ROUTING DECISION",
)) border_style="bright_blue",
))
console.print("[bold cyan]Why this model?[/bold cyan]") console.print("[bold cyan]Why this model?[/bold cyan]")
for r in decision.reasons: for r in decision.reasons:
console.print(f" [green]+[/green] {r}") console.print(f" [green]+[/green] {r}")
if decision.rejected_candidates: if decision.rejected_candidates:
console.print("\n[bold red]Rejected Candidates:[/bold red]") console.print("\n[bold red]Rejected Candidates:[/bold red]")
for cid, reason in decision.rejected_candidates.items(): for cid, reason in decision.rejected_candidates.items():
console.print(f" [dim]- {cid}: {reason}[/dim]") console.print(f" [dim]- {cid}: {reason}[/dim]")
_render_output(json_output, decision, _render)
asyncio.run(_route()) asyncio.run(_route())
@@ -174,6 +212,7 @@ def route(
def run( def run(
prompt: str = typer.Argument(..., help="Prompt to route and execute"), prompt: str = typer.Argument(..., help="Prompt to route and execute"),
policy: str = typer.Option("balanced", "--policy", "-p", help="Routing policy to use"), policy: str = typer.Option("balanced", "--policy", "-p", help="Routing policy to use"),
json_output: bool = typer.Option(False, "--json", "-j", help="Output raw JSON instead of formatted tables"),
): ):
"""Route request and execute inference through the chosen provider.""" """Route request and execute inference through the chosen provider."""
async def _run(): async def _run():
@@ -191,7 +230,11 @@ def run(
policy_name=policy, policy_name=policy,
) )
console.print(f"[bold cyan]Routing to:[/bold cyan] {decision.selected_model} via provider: {decision.provider}...") routing_msg = f"[bold cyan]Routing to:[/bold cyan] {decision.selected_model} via provider: {decision.provider}..."
if json_output:
err_console.print(routing_msg)
else:
console.print(routing_msg)
resp, fallback_used, orig_m, fb_reason = await execute_with_fallback( resp, fallback_used, orig_m, fb_reason = await execute_with_fallback(
prompt=prompt, prompt=prompt,
@@ -200,12 +243,32 @@ def run(
all_models=models, all_models=models,
) )
console.print(Panel( def _render() -> None:
resp.content, console.print(Panel(
title=f"RESPONSE from {resp.model} ({'FALLBACK: ' + orig_m if fallback_used else 'PRIMARY'})", resp.content,
subtitle=f"Latency: {resp.provider_latency_ms:.0f}ms | Tokens: {resp.total_tokens}", title=f"RESPONSE from {resp.model} ({'FALLBACK: ' + orig_m if fallback_used else 'PRIMARY'})",
border_style="green" if not fallback_used else "yellow", subtitle=f"Latency: {resp.provider_latency_ms:.0f}ms | Tokens: {resp.total_tokens}",
)) border_style="green" if not fallback_used else "yellow",
))
data = {
"decision": json.loads(decision.model_dump_json()) if hasattr(decision, "model_dump_json") else (decision.model_dump() if hasattr(decision, "model_dump") else decision),
"response": json.loads(resp.model_dump_json()) if hasattr(resp, "model_dump_json") else {
"content": getattr(resp, "content", ""),
"model": getattr(resp, "model", ""),
"provider_latency_ms": getattr(resp, "provider_latency_ms", 0),
"total_tokens": getattr(resp, "total_tokens", 0),
},
"fallback": {
"used": fallback_used,
"original_model": orig_m,
"reason": fb_reason,
}
}
if hasattr(resp, "model_dump") and not hasattr(resp, "model_dump_json"):
data["response"] = resp.model_dump()
_render_output(json_output, data, _render)
asyncio.run(_run()) asyncio.run(_run())
@@ -245,25 +308,66 @@ def traffic(limit: int = 10):
@app.command() @app.command()
def analytics(): def analytics(
json_output: bool = typer.Option(False, "--json", "-j", help="Output raw JSON instead of formatted tables"),
):
"""Display system-wide routing performance and cost savings analytics.""" """Display system-wide routing performance and cost savings analytics."""
async def _analytics(): async def _analytics():
await init_db() await init_db()
from app.analytics.service import get_system_analytics from app.analytics.service import get_system_analytics
async with AsyncSessionLocal() as db: async with AsyncSessionLocal() as db:
data = await get_system_analytics(db) data = await get_system_analytics(db)
console.print(Panel.fit(
f"[bold cyan]Total Routed Requests:[/bold cyan] {data['total_requests']}\n" def _render() -> None:
f"[bold green]Average Latency:[/bold green] {data['avg_latency_ms']}ms (Routing overhead: {data['avg_routing_latency_ms']}ms)\n" console.print(Panel.fit(
f"[bold yellow]Total Cost:[/bold yellow] ${data['total_cost_usd']:.6f}\n" f"[bold cyan]Total Routed Requests:[/bold cyan] {data['total_requests']}\n"
f"[bold magenta]Cost Saved vs Baseline:[/bold magenta] ${data['savings']['cost_saved_usd']:.6f} ({data['savings']['savings_percentage']}%)\n" f"[bold green]Average Latency:[/bold green] {data['avg_latency_ms']}ms (Routing overhead: {data['avg_routing_latency_ms']}ms)\n"
f"[bold blue]Fallback Rate:[/bold blue] {data['fallback_rate_percent']}%\n", f"[bold yellow]Total Cost:[/bold yellow] ${data['total_cost_usd']:.6f}\n"
title="MODEL ROUTER ANALYTICS OVERVIEW", f"[bold magenta]Cost Saved vs Baseline:[/bold magenta] ${data['savings']['cost_saved_usd']:.6f} ({data['savings']['savings_percentage']}%)\n"
border_style="bright_blue", f"[bold blue]Fallback Rate:[/bold blue] {data['fallback_rate_percent']}%\n",
)) title="MODEL ROUTER ANALYTICS OVERVIEW",
border_style="bright_blue",
))
_render_output(json_output, data, _render)
asyncio.run(_analytics()) asyncio.run(_analytics())
@app.command()
def ui(
host: str = typer.Option("127.0.0.1", "--host", "-h", help="Host address to bind the web server"),
port: int = typer.Option(8000, "--port", "-p", help="Port to run the Model Router Control Room on"),
open_browser: bool = typer.Option(False, "--open-browser", help="Automatically open dashboard in default web browser"),
):
"""Launch the Model Router API Gateway & AI Traffic Control Room UI."""
import uvicorn
import webbrowser
import threading
import time
url = f"http://{host}:{port}"
message = (
f"[bold cyan]Starting Model Router AI Traffic Control Room[/bold cyan]\n"
f"Web Dashboard: [bold green]{url}[/bold green]\n"
f"API Documentation: [bold green]{url}/docs[/bold green]"
)
console.print(Panel.fit(
message,
title="MODEL ROUTER UI LAUNCHER",
border_style="cyan"
))
if open_browser:
def _open():
time.sleep(1.2)
webbrowser.open(url)
threading.Thread(target=_open, daemon=True).start()
from app.server import app as fastapi_app
uvicorn.run(fastapi_app, host=host, port=port, log_level="info")
if __name__ == "__main__": if __name__ == "__main__":
app() app()
+2 -2
View File
@@ -1,4 +1,4 @@
import json import json
import logging import logging
from typing import Dict, Any, Optional from typing import Dict, Any, Optional
import datetime import datetime
@@ -27,7 +27,7 @@ def log_router_event(
payload = { payload = {
"event": event_name, "event": event_name,
"request_id": request_id, "request_id": request_id,
"timestamp": datetime.datetime.utcnow().isoformat(), "timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(),
"model": model, "model": model,
"provider": provider, "provider": provider,
"duration_ms": duration_ms, "duration_ms": duration_ms,
+2 -2
View File
@@ -1,4 +1,4 @@
import uuid import uuid
import datetime import datetime
from typing import List, Dict, Any, Optional from typing import List, Dict, Any, Optional
from app.models.schemas import ( from app.models.schemas import (
@@ -64,7 +64,7 @@ def route_request(
) -> RoutingDecision: ) -> RoutingDecision:
req_id = request_id or str(uuid.uuid4()) req_id = request_id or str(uuid.uuid4())
decision_id = f"dec_{uuid.uuid4().hex[:12]}" decision_id = f"dec_{uuid.uuid4().hex[:12]}"
now_iso = datetime.datetime.utcnow().isoformat() now_iso = datetime.datetime.now(datetime.timezone.utc).isoformat()
# 1. Rule evaluation # 1. Rule evaluation
rule_action, rule_target, rule_name = None, None, None rule_action, rule_target, rule_name = None, None, None
+111
View File
@@ -0,0 +1,111 @@
import asyncio
import json
import os
from contextlib import asynccontextmanager
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse
from sse_starlette.sse import EventSourceResponse
from sqlalchemy import select, desc
from app.config.settings import get_settings
from app.storage.database import init_db, AsyncSessionLocal
from app.storage.models import RequestRecord
from app.api.routes import router as api_router
settings = get_settings()
@asynccontextmanager
async def lifespan(app: FastAPI):
# Initialize SQLite database and seed defaults
await init_db()
yield
app = FastAPI(
title="Model Router",
description="Intelligent LLM Request Routing Platform — AI Traffic Control Room",
version="1.0.0",
lifespan=lifespan,
)
# CORS configuration
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(api_router)
@app.get("/api/traffic/stream")
async def stream_live_traffic():
"""
SSE Live Stream endpoint for the Traffic Control Room visualization.
Pushes newly persisted requests to connected frontend clients.
"""
async def event_generator():
last_seen_id = None
while True:
try:
async with AsyncSessionLocal() as session:
res = await session.execute(
select(RequestRecord).order_by(desc(RequestRecord.timestamp)).limit(1)
)
latest = res.scalar_one_or_none()
if latest and latest.request_id != last_seen_id:
last_seen_id = latest.request_id
payload = {
"request_id": latest.request_id,
"timestamp": latest.timestamp.isoformat() if latest.timestamp else "",
"prompt": latest.prompt,
"task_type": latest.task_type,
"complexity": latest.complexity,
"selected_model": latest.selected_model,
"provider": latest.provider,
"status": latest.status,
"total_latency_ms": latest.total_latency_ms,
"estimated_cost": latest.estimated_cost,
}
yield {
"event": "traffic_event",
"data": json.dumps(payload),
}
except Exception:
pass
await asyncio.sleep(1.5)
return EventSourceResponse(event_generator())
# Serve Bundled Frontend (SPA) if static directory exists
STATIC_DIR = os.path.join(os.path.dirname(__file__), "static")
if os.path.exists(STATIC_DIR):
assets_dir = os.path.join(STATIC_DIR, "assets")
if os.path.exists(assets_dir):
app.mount("/assets", StaticFiles(directory=assets_dir), name="assets")
@app.get("/{full_path:path}")
async def serve_spa(full_path: str):
if full_path.startswith("api/") or full_path.startswith("docs") or full_path.startswith("openapi.json"):
raise HTTPException(status_code=404, detail="Not Found")
target_file = os.path.join(STATIC_DIR, full_path)
if os.path.exists(target_file) and os.path.isfile(target_file):
return FileResponse(target_file)
return FileResponse(os.path.join(STATIC_DIR, "index.html"))
else:
@app.get("/")
async def root():
return {
"message": "Model Router AI Control Room Backend is active.",
"docs": "/docs",
"health": "/api/health",
}
+35 -8
View File
@@ -1,9 +1,11 @@
import asyncio import asyncio
import json import json
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from fastapi import FastAPI from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from sse_starlette.sse import EventSourceResponse from sse_starlette.sse import EventSourceResponse
import uvicorn
from app.server import app, settings
from app.config.settings import get_settings from app.config.settings import get_settings
from app.storage.database import init_db, AsyncSessionLocal from app.storage.database import init_db, AsyncSessionLocal
@@ -80,15 +82,40 @@ async def stream_live_traffic():
return EventSourceResponse(event_generator()) return EventSourceResponse(event_generator())
@app.get("/") # Serve Bundled Frontend (SPA) if static directory exists
async def root(): import os
return { from fastapi.staticfiles import StaticFiles
"message": "Model Router AI Control Room Backend is active.", from fastapi.responses import FileResponse
"docs": "/docs", from fastapi import HTTPException
"health": "/api/health",
} STATIC_DIR = os.path.join(os.path.dirname(__file__), "app", "static")
if os.path.exists(STATIC_DIR):
assets_dir = os.path.join(STATIC_DIR, "assets")
if os.path.exists(assets_dir):
app.mount("/assets", StaticFiles(directory=assets_dir), name="assets")
@app.get("/{full_path:path}")
async def serve_spa(full_path: str):
if full_path.startswith("api/") or full_path.startswith("docs") or full_path.startswith("openapi.json"):
raise HTTPException(status_code=404, detail="Not Found")
target_file = os.path.join(STATIC_DIR, full_path)
if os.path.exists(target_file) and os.path.isfile(target_file):
return FileResponse(target_file)
return FileResponse(os.path.join(STATIC_DIR, "index.html"))
else:
@app.get("/")
async def root():
return {
"message": "Model Router AI Control Room Backend is active.",
"docs": "/docs",
"health": "/api/health",
}
if __name__ == "__main__": if __name__ == "__main__":
import uvicorn import uvicorn
uvicorn.run("main:app", host=settings.HOST, port=settings.PORT, reload=settings.DEBUG) uvicorn.run("main:app", host=settings.HOST, port=settings.PORT, reload=settings.DEBUG)
uvicorn.run("app.server:app", host=settings.HOST, port=settings.PORT, reload=settings.DEBUG)
+46
View File
@@ -0,0 +1,46 @@
import json
import pytest
from typer.testing import CliRunner
from app.cli.main import app
runner = CliRunner()
def test_cli_doctor_json():
result = runner.invoke(app, ["doctor", "--json"])
assert result.exit_code == 0
data = json.loads(result.stdout)
assert "python" in data
assert "database" in data
assert "mock_engine" in data
def test_cli_models_json():
result = runner.invoke(app, ["models", "--json"])
assert result.exit_code == 0
data = json.loads(result.stdout)
assert isinstance(data, list)
if len(data) > 0:
assert "id" in data[0]
assert "provider" in data[0]
def test_cli_route_json():
result = runner.invoke(app, ["route", "test prompt", "--json"])
assert result.exit_code == 0
data = json.loads(result.stdout)
assert "selected_model" in data
assert "estimated_latency_ms" in data
def test_cli_run_json():
result = runner.invoke(app, ["run", "test prompt", "--json"])
assert result.exit_code == 0
data = json.loads(result.stdout)
assert "decision" in data
assert "response" in data
assert "fallback" in data
def test_cli_analytics_json():
result = runner.invoke(app, ["analytics", "--json"])
assert result.exit_code == 0
data = json.loads(result.stdout)
assert "total_requests" in data
assert "avg_latency_ms" in data
assert "savings" in data
+61
View File
@@ -64,3 +64,64 @@ async def test_api_analytics():
data = res.json() data = res.json()
assert "total_requests" in data assert "total_requests" in data
assert "savings" in data assert "savings" in data
@pytest.mark.asyncio
async def test_api_traffic_export_json_contains_audit_fields():
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
generate_res = await client.post(
"/api/generate",
json={"prompt": "Write a Python function for a traffic export"},
)
assert generate_res.status_code == 200
export_res = await client.get("/api/traffic/export?format=json")
assert export_res.status_code == 200
assert export_res.headers["content-type"].startswith("application/json")
assert "attachment" in export_res.headers["content-disposition"]
records = export_res.json()
assert records
assert {
"timestamp",
"request_id",
"prompt_preview",
"task_type",
"complexity",
"selected_model",
"input_tokens",
"output_tokens",
"total_tokens",
"cost_saved",
"total_latency_ms",
}.issubset(records[0])
@pytest.mark.asyncio
async def test_api_traffic_export_csv_returns_downloadable_rows():
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
generate_res = await client.post(
"/api/generate",
json={"prompt": "Summarize this traffic record"},
)
assert generate_res.status_code == 200
export_res = await client.get("/api/traffic/export?format=csv")
assert export_res.status_code == 200
assert export_res.headers["content-type"].startswith("text/csv")
assert "traffic-export.csv" in export_res.headers["content-disposition"]
lines = export_res.text.splitlines()
assert lines[0].startswith("timestamp,request_id,prompt_preview,")
assert len(lines) >= 2
@pytest.mark.asyncio
async def test_api_traffic_export_rejects_unknown_format():
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
export_res = await client.get("/api/traffic/export?format=xml")
assert export_res.status_code == 422
File diff suppressed because it is too large Load Diff
+46
View File
@@ -0,0 +1,46 @@
import React, { createContext, useContext, useEffect, useState } from 'react';
type Theme = 'light' | 'dark';
interface ThemeContextType {
theme: Theme;
toggleTheme: () => void;
}
const ThemeContext = createContext<ThemeContextType | undefined>(undefined);
export const ThemeProvider: React.FC<{ children: React.ReactNode }> = ({ children }) => {
const [theme, setTheme] = useState<Theme>(() => {
const saved = localStorage.getItem('theme');
if (saved === 'light' || saved === 'dark') return saved;
if (window.matchMedia && window.matchMedia('(prefers-color-scheme: light)').matches) {
return 'light';
}
return 'dark';
});
useEffect(() => {
const root = window.document.documentElement;
root.classList.remove('light', 'dark');
root.classList.add(theme);
localStorage.setItem('theme', theme);
}, [theme]);
const toggleTheme = () => {
setTheme(prev => (prev === 'dark' ? 'light' : 'dark'));
};
return (
<ThemeContext.Provider value={{ theme, toggleTheme }}>
{children}
</ThemeContext.Provider>
);
};
export const useTheme = () => {
const context = useContext(ThemeContext);
if (context === undefined) {
throw new Error('useTheme must be used within a ThemeProvider');
}
return context;
};
+19 -7
View File
@@ -1,4 +1,4 @@
import React from 'react'; import React from 'react';
import { import {
Activity, Activity,
Compass, Compass,
@@ -6,8 +6,11 @@ import {
Cpu, Cpu,
BarChart3, BarChart3,
Sliders, Sliders,
Radio Radio,
Sun,
Moon
} from 'lucide-react'; } from 'lucide-react';
import { useTheme } from '../ThemeContext';
interface NavbarProps { interface NavbarProps {
activeTab: string; activeTab: string;
@@ -16,6 +19,7 @@ interface NavbarProps {
} }
export const Navbar: React.FC<NavbarProps> = ({ activeTab, setActiveTab, onlineStatus = true }) => { export const Navbar: React.FC<NavbarProps> = ({ activeTab, setActiveTab, onlineStatus = true }) => {
const { theme, toggleTheme } = useTheme();
const navItems = [ const navItems = [
{ id: 'dashboard', label: 'Control Room', icon: Activity }, { id: 'dashboard', label: 'Control Room', icon: Activity },
{ id: 'playground', label: 'Playground', icon: Compass }, { id: 'playground', label: 'Playground', icon: Compass },
@@ -27,7 +31,7 @@ export const Navbar: React.FC<NavbarProps> = ({ activeTab, setActiveTab, onlineS
]; ];
return ( return (
<header className="sticky top-0 z-50 w-full border-b border-[#1F2833] bg-[#07090C]/90 backdrop-blur-md"> <header className="sticky top-0 z-50 w-full border-b border-border-subtle bg-bg-primary/90 backdrop-blur-md">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 h-16 flex items-center justify-between gap-4"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 h-16 flex items-center justify-between gap-4">
{/* Brand / Logo */} {/* Brand / Logo */}
@@ -35,7 +39,7 @@ export const Navbar: React.FC<NavbarProps> = ({ activeTab, setActiveTab, onlineS
onClick={() => setActiveTab('dashboard')} onClick={() => setActiveTab('dashboard')}
className="flex items-center gap-3 cursor-pointer select-none shrink-0 group" className="flex items-center gap-3 cursor-pointer select-none shrink-0 group"
> >
<div className="w-10 h-10 rounded-xl bg-[#0D1117] border border-accent-primary/40 flex items-center justify-center overflow-hidden shadow-[0_0_18px_rgba(56,148,255,0.25)] group-hover:border-accent-primary transition-all p-1"> <div className="w-10 h-10 rounded-xl bg-bg-secondary border border-accent-primary/40 flex items-center justify-center overflow-hidden shadow-[0_0_18px_rgba(56,148,255,0.25)] group-hover:border-accent-primary transition-all p-1">
<img <img
src="/logo.png" src="/logo.png"
alt="Model Router Logo" alt="Model Router Logo"
@@ -59,7 +63,7 @@ export const Navbar: React.FC<NavbarProps> = ({ activeTab, setActiveTab, onlineS
</div> </div>
{/* Navigation Tabs */} {/* Navigation Tabs */}
<nav className="hidden lg:flex items-center gap-1.5 p-1 rounded-xl bg-[#0D1117]/80 border border-[#1F2833]/80"> <nav className="hidden lg:flex items-center gap-1.5 p-1 rounded-xl bg-bg-secondary/80 border border-border-subtle/80">
{navItems.map((item) => { {navItems.map((item) => {
const Icon = item.icon; const Icon = item.icon;
const isActive = activeTab === item.id; const isActive = activeTab === item.id;
@@ -70,7 +74,7 @@ export const Navbar: React.FC<NavbarProps> = ({ activeTab, setActiveTab, onlineS
className={`flex items-center gap-2 px-3 py-1.5 rounded-lg text-xs font-mono font-semibold transition-all duration-150 whitespace-nowrap ${ className={`flex items-center gap-2 px-3 py-1.5 rounded-lg text-xs font-mono font-semibold transition-all duration-150 whitespace-nowrap ${
isActive isActive
? 'bg-accent-primary text-bg-primary shadow-[0_0_12px_rgba(56,148,255,0.4)] font-bold' ? 'bg-accent-primary text-bg-primary shadow-[0_0_12px_rgba(56,148,255,0.4)] font-bold'
: 'text-text-muted hover:text-text-primary hover:bg-[#141920]' : 'text-text-muted hover:text-text-primary hover:bg-bg-card'
}`} }`}
> >
<Icon className={`w-3.5 h-3.5 ${isActive ? 'text-bg-primary stroke-[2.5]' : 'text-text-muted stroke-[2]'}`} /> <Icon className={`w-3.5 h-3.5 ${isActive ? 'text-bg-primary stroke-[2.5]' : 'text-text-muted stroke-[2]'}`} />
@@ -104,7 +108,15 @@ export const Navbar: React.FC<NavbarProps> = ({ activeTab, setActiveTab, onlineS
{/* Status Indicator */} {/* Status Indicator */}
<div className="hidden sm:flex items-center gap-2 shrink-0"> <div className="hidden sm:flex items-center gap-2 shrink-0">
<div className="flex items-center gap-2 px-3 py-1.5 rounded-xl bg-[#0D1117] border border-[#1F2833] font-mono text-xs shadow-inner"> <button
onClick={toggleTheme}
className="p-1.5 rounded-xl bg-bg-secondary border border-border-subtle text-text-muted hover:text-text-primary hover:border-border-active transition-all"
title="Toggle theme"
>
{theme === 'dark' ? <Sun className="w-4 h-4" /> : <Moon className="w-4 h-4" />}
</button>
<div className="flex items-center gap-2 px-3 py-1.5 rounded-xl bg-bg-secondary border border-border-subtle font-mono text-xs shadow-inner">
<span className={`w-2 h-2 rounded-full ${onlineStatus ? 'bg-accent-success shadow-[0_0_8px_#10B981]' : 'bg-accent-error'}`} /> <span className={`w-2 h-2 rounded-full ${onlineStatus ? 'bg-accent-success shadow-[0_0_8px_#10B981]' : 'bg-accent-error'}`} />
<span className="text-[11px] font-bold tracking-wide text-text-primary whitespace-nowrap"> <span className="text-[11px] font-bold tracking-wide text-text-primary whitespace-nowrap">
ROUTER READY ROUTER READY
+43 -19
View File
@@ -1,17 +1,17 @@
@import "tailwindcss"; @import "tailwindcss";
@theme { @theme {
--color-bg-primary: #07090C; --color-bg-primary: var(--theme-bg-primary);
--color-bg-secondary: #0D1117; --color-bg-secondary: var(--theme-bg-secondary);
--color-bg-card: #121820; --color-bg-card: var(--theme-bg-card);
--color-bg-card-hover: #17202B; --color-bg-card-hover: var(--theme-bg-card-hover);
--color-border-subtle: #1F2833; --color-border-subtle: var(--theme-border-subtle);
--color-border-active: #324050; --color-border-active: var(--theme-border-active);
--color-text-primary: #F0F4F8; --color-text-primary: var(--theme-text-primary);
--color-text-muted: #8B98A5; --color-text-muted: var(--theme-text-muted);
--color-text-dim: #546270; --color-text-dim: var(--theme-text-dim);
--color-accent-primary: #3894FF; --color-accent-primary: #3894FF;
--color-accent-secondary: #F59E0B; --color-accent-secondary: #F59E0B;
@@ -24,19 +24,43 @@
--font-sans: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; --font-sans: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
} }
:root {
--theme-bg-primary: #F8FAFC;
--theme-bg-secondary: #F1F5F9;
--theme-bg-card: #FFFFFF;
--theme-bg-card-hover: #F8FAFC;
--theme-border-subtle: #E2E8F0;
--theme-border-active: #CBD5E1;
--theme-text-primary: #0F172A;
--theme-text-muted: #475569;
--theme-text-dim: #64748B;
}
.dark {
--theme-bg-primary: #07090C;
--theme-bg-secondary: #0D1117;
--theme-bg-card: #121820;
--theme-bg-card-hover: #17202B;
--theme-border-subtle: #1F2833;
--theme-border-active: #324050;
--theme-text-primary: #F0F4F8;
--theme-text-muted: #8B98A5;
--theme-text-dim: #546270;
}
body { body {
background-color: #07090C; background-color: var(--theme-bg-primary);
color: #F0F4F8; color: var(--theme-text-primary);
font-family: 'Inter', -apple-system, BlinkMacSystemFont, sans-serif; font-family: 'Inter', -apple-system, BlinkMacSystemFont, sans-serif;
overflow-x: hidden; overflow-x: hidden;
} }
/* Custom glow borders and glass effects */ /* Custom glow borders and glass effects */
.glass-panel { .glass-panel {
background: rgba(18, 24, 32, 0.75); background: color-mix(in srgb, var(--theme-bg-card) 75%, transparent);
backdrop-filter: blur(12px); backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px); -webkit-backdrop-filter: blur(12px);
border: 1px solid rgba(40, 52, 68, 0.6); border: 1px solid color-mix(in srgb, var(--theme-border-active) 60%, transparent);
} }
.glass-panel-hover:hover { .glass-panel-hover:hover {
@@ -48,8 +72,8 @@ body {
.grid-bg { .grid-bg {
background-size: 28px 28px; background-size: 28px 28px;
background-image: background-image:
linear-gradient(to right, rgba(31, 40, 51, 0.4) 1px, transparent 1px), linear-gradient(to right, color-mix(in srgb, var(--theme-border-subtle) 40%, transparent) 1px, transparent 1px),
linear-gradient(to bottom, rgba(31, 40, 51, 0.4) 1px, transparent 1px); linear-gradient(to bottom, color-mix(in srgb, var(--theme-border-subtle) 40%, transparent) 1px, transparent 1px);
} }
.radar-glow { .radar-glow {
@@ -83,12 +107,12 @@ body {
height: 6px; height: 6px;
} }
::-webkit-scrollbar-track { ::-webkit-scrollbar-track {
background: #07090C; background: var(--theme-bg-primary);
} }
::-webkit-scrollbar-thumb { ::-webkit-scrollbar-thumb {
background: #1F2833; background: var(--theme-border-subtle);
border-radius: 3px; border-radius: 3px;
} }
::-webkit-scrollbar-thumb:hover { ::-webkit-scrollbar-thumb:hover {
background: #324050; background: var(--theme-border-active);
} }
+5 -2
View File
@@ -1,10 +1,13 @@
import React from 'react' import React from 'react'
import ReactDOM from 'react-dom/client' import ReactDOM from 'react-dom/client'
import App from './App' import App from './App'
import { ThemeProvider } from './ThemeContext'
import './index.css' import './index.css'
ReactDOM.createRoot(document.getElementById('root')!).render( ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode> <React.StrictMode>
<App /> <ThemeProvider>
<App />
</ThemeProvider>
</React.StrictMode>, </React.StrictMode>,
) )
+22 -10
View File
@@ -1,8 +1,10 @@
import React, { useEffect, useState } from 'react'; import React, { useEffect, useState } from 'react';
import { Sliders, Shield, DollarSign, Cpu, CheckCircle } from 'lucide-react'; import { Sliders, Shield, DollarSign, Cpu, CheckCircle, Sun, Moon } from 'lucide-react';
import { fetchApi } from '../lib/api'; import { fetchApi } from '../lib/api';
import { useTheme } from '../ThemeContext';
export const SettingsPage: React.FC = () => { export const SettingsPage: React.FC = () => {
const { theme, toggleTheme } = useTheme();
const [budget, setBudget] = useState<{ const [budget, setBudget] = useState<{
daily_limit: number; daily_limit: number;
monthly_limit: number; monthly_limit: number;
@@ -17,14 +19,24 @@ export const SettingsPage: React.FC = () => {
return ( return (
<div className="space-y-8 max-w-4xl mx-auto pb-16 font-mono"> <div className="space-y-8 max-w-4xl mx-auto pb-16 font-mono">
<div className="border-b border-border-subtle pb-4"> <div className="border-b border-border-subtle pb-4 flex justify-between items-start">
<h2 className="text-xl font-bold text-text-primary tracking-wide flex items-center gap-2"> <div>
<Sliders className="w-5 h-5 text-accent-primary" /> <h2 className="text-xl font-bold text-text-primary tracking-wide flex items-center gap-2">
SYSTEM CONFIGURATION & BUDGET GUARDS <Sliders className="w-5 h-5 text-accent-primary" />
</h2> SYSTEM CONFIGURATION & BUDGET GUARDS
<p className="text-xs text-text-muted mt-1"> </h2>
Configure daily and monthly spend limits, routing threshold interventions, and environment parameters. <p className="text-xs text-text-muted mt-1">
</p> Configure daily and monthly spend limits, routing threshold interventions, and environment parameters.
</p>
</div>
<button
onClick={toggleTheme}
className="flex items-center gap-2 px-3 py-1.5 rounded-lg bg-bg-secondary border border-border-subtle text-text-muted hover:text-text-primary hover:border-border-active transition-all text-xs font-bold"
>
{theme === 'dark' ? <Sun className="w-4 h-4" /> : <Moon className="w-4 h-4" />}
{theme === 'dark' ? 'LIGHT MODE' : 'DARK MODE'}
</button>
</div> </div>
{/* Budget Management */} {/* Budget Management */}
+33 -7
View File
@@ -1,11 +1,12 @@
import React, { useEffect, useState } from 'react'; import React, { useEffect, useState } from 'react';
import { Radio, RefreshCw, Layers, Clock, DollarSign, Cpu } from 'lucide-react'; import { Download, Radio, RefreshCw } from 'lucide-react';
import { fetchApi } from '../lib/api'; import { fetchApi } from '../lib/api';
import { TrafficItem } from '../types'; import { TrafficItem } from '../types';
export const Traffic: React.FC = () => { export const Traffic: React.FC = () => {
const [traffic, setTraffic] = useState<TrafficItem[]>([]); const [traffic, setTraffic] = useState<TrafficItem[]>([]);
const [loading, setLoading] = useState(false); const [loading, setLoading] = useState(false);
const [exportFormat, setExportFormat] = useState<'csv' | 'json'>('csv');
const loadTraffic = async () => { const loadTraffic = async () => {
setLoading(true); setLoading(true);
@@ -19,6 +20,13 @@ export const Traffic: React.FC = () => {
} }
}; };
const exportTraffic = () => {
window.open(
`http://localhost:8000/api/traffic/export?format=${exportFormat}`,
'_blank',
);
};
useEffect(() => { useEffect(() => {
loadTraffic(); loadTraffic();
const eventSource = new EventSource('http://localhost:8000/api/traffic/stream'); const eventSource = new EventSource('http://localhost:8000/api/traffic/stream');
@@ -49,12 +57,30 @@ export const Traffic: React.FC = () => {
</p> </p>
</div> </div>
<button <div className="flex items-center gap-2">
onClick={loadTraffic} <select
className="p-2 rounded-lg border border-border-subtle bg-bg-card text-text-muted hover:text-text-primary font-mono text-xs flex items-center gap-1.5" value={exportFormat}
> onChange={(event) => setExportFormat(event.target.value as 'csv' | 'json')}
<RefreshCw className={`w-3.5 h-3.5 ${loading ? 'animate-spin' : ''}`} /> Refresh aria-label="Export format"
</button> className="rounded-lg border border-border-subtle bg-bg-card px-2 py-2 text-xs font-mono text-text-muted outline-none"
>
<option value="csv">CSV</option>
<option value="json">JSON</option>
</select>
<button
onClick={exportTraffic}
className="p-2 rounded-lg border border-border-subtle bg-bg-card text-text-muted hover:text-text-primary font-mono text-xs flex items-center gap-1.5"
title="Export telemetry"
>
<Download className="w-3.5 h-3.5" /> Export Telemetry
</button>
<button
onClick={loadTraffic}
className="p-2 rounded-lg border border-border-subtle bg-bg-card text-text-muted hover:text-text-primary font-mono text-xs flex items-center gap-1.5"
>
<RefreshCw className={`w-3.5 h-3.5 ${loading ? 'animate-spin' : ''}`} /> Refresh
</button>
</div>
</div> </div>
<div className="rounded-xl border border-border-subtle bg-bg-card p-5 space-y-4 overflow-x-auto"> <div className="rounded-xl border border-border-subtle bg-bg-card p-5 space-y-4 overflow-x-auto">
+58
View File
@@ -0,0 +1,58 @@
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "model-router-cli"
version = "1.0.2"
description = "Intelligent, explainable LLM request routing platform & AI Traffic Control Room"
readme = "README.md"
requires-python = ">=3.10"
authors = [
{ name = "PicadoLabs", email = "picadolabs@gmail.com" }
]
maintainers = [
{ name = "Kaap10", email = "picadolabs@gmail.com" }
]
license = { text = "MIT" }
keywords = ["llm", "ai", "router", "fastapi", "ollama", "openai", "cost-optimization"]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
"Topic :: Software Development :: Libraries :: Application Frameworks",
]
dependencies = [
"fastapi>=0.110.0",
"uvicorn[standard]>=0.28.0",
"pydantic>=2.6.0",
"pydantic-settings>=2.2.0",
"sqlalchemy>=2.0.28",
"aiosqlite>=0.20.0",
"httpx>=0.27.0",
"typer>=0.12.0",
"rich>=13.7.0",
"sse-starlette>=2.0.0",
"python-multipart>=0.0.9",
"python-dotenv>=1.0.1",
"tiktoken>=0.6.0"
]
[project.urls]
Homepage = "https://picadolabs.me"
Repository = "https://github.com/PicadoLabs/ai-model-router"
Issues = "https://github.com/PicadoLabs/ai-model-router/issues"
[project.scripts]
modelrouter = "app.cli.main:app"
[tool.setuptools.packages.find]
where = ["backend"]
[tool.setuptools.package-data]
"app" = ["static/**/*", "static/*"]
+51
View File
@@ -0,0 +1,51 @@
import os
import sys
import shutil
import subprocess
def build():
print("==================================================")
print("STEP 1: Compiling React Frontend (Traffic Control Room)")
print("==================================================")
frontend_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "frontend"))
result = subprocess.run(["npm", "run", "build"], cwd=frontend_dir, shell=True)
if result.returncode != 0:
print("[ERROR] Frontend build failed!")
sys.exit(1)
print("\n==================================================")
print("STEP 2: Bundling UI Assets into backend/app/static")
print("==================================================")
dist_dir = os.path.join(frontend_dir, "dist")
static_dest = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "backend", "app", "static"))
if os.path.exists(static_dest):
shutil.rmtree(static_dest)
shutil.copytree(dist_dir, static_dest)
print(f"[OK] Successfully copied static UI files from {dist_dir} to {static_dest}")
print("\n==================================================")
print("STEP 3: Building Python Distribution (.tar.gz and .whl)")
print("==================================================")
root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
# Ensure build is installed
subprocess.run([sys.executable, "-m", "pip", "install", "--upgrade", "build", "twine", "wheel"], cwd=root_dir, check=True)
# Build package
build_result = subprocess.run([sys.executable, "-m", "build"], cwd=root_dir)
if build_result.returncode != 0:
print("[ERROR] Python build failed!")
sys.exit(1)
print("\n==================================================")
print("STEP 4: Validating Artifacts with Twine Check")
print("==================================================")
dist_artifacts = os.path.join(root_dir, "dist", "*")
subprocess.run([sys.executable, "-m", "twine", "check", dist_artifacts], cwd=root_dir, shell=True)
print("\n[SUCCESS] ai-model-router package successfully built!")
print(f"Artifacts ready in: {os.path.join(root_dir, 'dist')}")
if __name__ == "__main__":
build()