feat: complete production-quality Model Router AI traffic control platform

This commit is contained in:
Vardhman Gupta
2026-08-31 17:33:38 +05:30
commit 4e6f742272
66 changed files with 8969 additions and 0 deletions
+92
View File
@@ -0,0 +1,92 @@
# Model Router
Intelligent, explainable, cost- and latency-aware LLM request routing platform and AI Traffic Control Room.
---
## Overview
Model Router intercepts incoming AI requests, analyzes their task type and complexity, evaluates available models against a configurable multi-criteria scoring objective, selects the optimal candidate, and dispatches the request with automatic fallback handling and budget guards.
The platform is designed local-first, allowing full local development and testing using Mock models or local Ollama instances without requiring paid external API keys.
---
## Core Capabilities
- Dual-Mode Request Analyzer: Deterministic heuristics (<3ms latency overhead) for 12 task types, continuous complexity scoring (0.0 to 1.0), and requirement detection, plus an optional LLM classifier mode.
- Explainable Routing Engine: Multi-criteria weighted scoring across Quality, Cost Efficiency, Speed, Capabilities, and Reliability with transparent decision factor reports and candidate rejection logs.
- Provider Abstraction: Decoupled adapters for Mock (simulation), Ollama (local), OpenAI, Anthropic, and Google Gemini.
- Resilience and Fallback: Automated retry classification for transient errors (timeouts, HTTP 429, 503) and tiered fallback to local/mock alternatives.
- Budget Guards: Real-time spend tracking with automated threshold interventions (80% cost optimization, 95% local-only saver, 100% block).
- Traffic Control Room UI: Real-time dark operational interface featuring live topology graphs, playground inspector, SSE live request stream, visual rules builder, and cost savings simulator.
- Developer CLI: Terminal diagnostics (`modelrouter doctor`), routing dry-run (`route`), execution (`run`), model catalog (`models`), and analytics (`analytics`).
---
## Quickstart
### 1. Prerequisites
- Python 3.10+
- Node.js 18+ and npm
### 2. Backend Setup
```bash
# Clone the repository
git clone https://github.com/your-org/model-router.git
cd model-router
# Install dependencies
pip install -r requirements.txt
# Run system diagnostics
python backend/app/cli/main.py doctor
# Start API server (port 8000)
python backend/main.py
```
### 3. Frontend Setup
```bash
cd frontend
npm install
npm run dev
# Access UI at http://localhost:5173
```
---
## CLI Usage Examples
```bash
# Inspect routing decision for a coding prompt (Dry Run)
python backend/app/cli/main.py route "Write a Python function to parse JSON"
# Route and execute a complex debugging query
python backend/app/cli/main.py run "Debug this distributed async deadlock in high-throughput worker pool"
# List active models in registry
python backend/app/cli/main.py models
# View aggregate system analytics
python backend/app/cli/main.py analytics
```
---
## Testing & Build Verification
```bash
# Run backend test suite (15 unit, integration, and e2e tests)
pytest backend/tests
# Build frontend production bundle
cd frontend
npm run build
```
---
## License
MIT License. Open-source software.