github/ai-model-router
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: establish PicadoLabs open-source governance, CI workflows, and contributor guidelines

Browse Source
This commit is contained in:
Vardhman Gupta
2026-09-09 17:59:46 +05:30
parent 4e6f742272
commit 775645f600
11 changed files with 1759 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+233 -38
View File
@@ -1,4 +1,4 @@
# Model Router
# Model Router
Intelligent, explainable, cost- and latency-aware LLM request routing platform and AI Traffic Control Room.
@@ -6,87 +6,282 @@ Intelligent, explainable, cost- and latency-aware LLM request routing platform a
## Overview
Model Router intercepts incoming AI requests, analyzes their task type and complexity, evaluates available models against a configurable multi-criteria scoring objective, selects the optimal candidate, and dispatches the request with automatic fallback handling and budget guards.
Model Router intercepts incoming AI requests, analyzes their task type and continuous complexity, evaluates available models against a configurable multi-criteria scoring objective, selects the optimal candidate, and dispatches the request with automatic fallback handling and budget guards.
The platform is designed local-first, allowing full local development and testing using Mock models or local Ollama instances without requiring paid external API keys.
---
## Core Capabilities
## Key Features
- Dual-Mode Request Analyzer: Deterministic heuristics (<3ms latency overhead) for 12 task types, continuous complexity scoring (0.0 to 1.0), and requirement detection, plus an optional LLM classifier mode.
- Explainable Routing Engine: Multi-criteria weighted scoring across Quality, Cost Efficiency, Speed, Capabilities, and Reliability with transparent decision factor reports and candidate rejection logs.
- Provider Abstraction: Decoupled adapters for Mock (simulation), Ollama (local), OpenAI, Anthropic, and Google Gemini.
- Resilience and Fallback: Automated retry classification for transient errors (timeouts, HTTP 429, 503) and tiered fallback to local/mock alternatives.
- Budget Guards: Real-time spend tracking with automated threshold interventions (80% cost optimization, 95% local-only saver, 100% block).
- Traffic Control Room UI: Real-time dark operational interface featuring live topology graphs, playground inspector, SSE live request stream, visual rules builder, and cost savings simulator.
- Developer CLI: Terminal diagnostics (`modelrouter doctor`), routing dry-run (`route`), execution (`run`), model catalog (`models`), and analytics (`analytics`).
- **Dual-Mode Request Analyzer**: Deterministic heuristics (<3ms latency overhead) for 12 task types, continuous complexity scoring (0.05 to 0.99), and requirement detection, plus an optional LLM classifier mode.
- **Explainable Routing Engine**: Multi-criteria weighted scoring across Quality, Cost Efficiency, Speed, Capabilities, and Reliability with transparent decision factor reports and candidate rejection logs.
- **Provider Abstraction**: Decoupled adapters for Mock (simulation), Ollama (local), OpenAI, Anthropic, and Google Gemini.
- **Resilience and Tiered Fallback**: Automated retry classification for transient errors (timeouts, HTTP 429, 503) and tiered fallback to local/mock alternatives.
- **Budget Control Guards**: Real-time spend tracking with automated threshold interventions (80% cost optimization, 95% local-only saver, 100% block).
- **Traffic Control Room UI**: Real-time dark operational interface featuring live topology graphs, playground inspector, SSE live request stream, visual rules builder, and cost savings simulator.
- **Developer CLI**: Terminal diagnostics (`doctor`), routing dry-run (`route`), execution (`run`), model catalog (`models`), and analytics (`analytics`).
---
## Architecture & Workflow
```text
[ Client / SDK / Typer CLI ]
│
▼
[ FastAPI Gateway (Port 8000) ]
│
┌────────┴──────────────────────────┐
│ 1. Request Analyzer (<3ms) │ --> Task Type, Complexity, Context Size
│ 2. Priority Rules Evaluation │ --> Conditional Overrides
│ 3. Candidate Hard Pruning │ --> Filter Ineligible Models (Context / Caps)
│ 4. Multi-Criteria Scoring │ --> Normalized 0-100 Score across 5 Dimensions
│ 5. Decision Factor Generator │ --> Itemized Explainability Breakdown
└────────┬──────────────────────────┘
│
▼
[ Fallback Supervisor & Provider Layer ]
├── Local: Ollama Provider (qwen2.5-coder, llama3.2, deepseek-r1)
├── Simulated: In-Memory Mock Provider (Zero Cost)
└── Cloud: OpenAI, Anthropic, Google Gemini (Optional)
│
▼
[ Storage & Observability Engine ]
├── Asynchronous SQLite WAL Database (`model_router.db`)
└── Server-Sent Events (SSE) Stream -> React Control Room (Port 5173)
```
---
## Supported Platforms & Prerequisites
### Supported Platforms
- Linux (Ubuntu 20.04+, Debian 11+, Fedora)
- macOS (macOS 12+ / Apple Silicon & Intel)
- Windows (Windows 10, Windows 11 / PowerShell & WSL2)
### Prerequisites
- Python 3.10, 3.11, or 3.12
- Node.js 18+ and npm
- (Optional) [Ollama](https://ollama.com/) for local model inference
---
## Installation & Setup
### 1. Clone the Repository
```bash
git clone https://github.com/PicadoLabs/AI-Model-Router.git
cd AI-Model-Router
```
### 2. Backend Installation
```bash
# Create and activate virtual environment
python -m venv venv
# On Linux/macOS:
source venv/bin/activate
# On Windows PowerShell:
.\venv\Scripts\Activate.ps1
# Install Python dependencies
pip install -r requirements.txt
# Create environment file from template
cp .env.example .env
```
### 3. Frontend Installation
```bash
cd frontend
npm install
cd ..
```
---
## Configuration & Environment Variables
Configuration is loaded via Pydantic Settings from the `.env` file:
| Variable | Default | Description |
| :--- | :--- | :--- |
| `APP_ENV` | `development` | Application environment (`development`, `production`, `test`) |
| `PORT` | `8000` | FastAPI server port |
| `HOST` | `0.0.0.0` | FastAPI server host |
| `DATABASE_URL` | `sqlite+aiosqlite:///./model_router.db` | SQLAlchemy database connection URI |
| `ROUTER_ANALYZER` | `rules` | Default analyzer mode (`rules` for heuristics, `llm` for model classifier) |
| `DEFAULT_ROUTING_POLICY` | `balanced` | Default routing weights (`balanced`, `lowest_cost`, `lowest_latency`, `highest_quality`) |
| `BASELINE_MODEL_ID` | `mock-power` | Reference model ID for calculating baseline cost savings |
| `DEFAULT_PROVIDER` | `mock` | Default execution provider (`mock`, `ollama`) |
| `OLLAMA_BASE_URL` | `http://localhost:11434` | Ollama HTTP endpoint |
| `OPENAI_API_KEY` | *(empty)* | Optional OpenAI API Key |
| `ANTHROPIC_API_KEY` | *(empty)* | Optional Anthropic API Key |
| `GEMINI_API_KEY` | *(empty)* | Optional Google Gemini API Key |
| `DAILY_BUDGET` | `10.00` | Daily spend limit in USD |
| `MONTHLY_BUDGET` | `100.00` | Monthly spend limit in USD |
| `MAX_RETRIES` | `2` | Maximum retries before triggering cascading fallback |
| `PROVIDER_TIMEOUT_SECONDS` | `30.0` | Provider HTTP timeout in seconds |
---
## Quickstart
### 1. Prerequisites
- Python 3.10+
- Node.js 18+ and npm
### 2. Backend Setup
### 1. Run System Diagnostics
```bash
# Clone the repository
git clone https://github.com/your-org/model-router.git
cd model-router
# Install dependencies
pip install -r requirements.txt
# Run system diagnostics
python backend/app/cli/main.py doctor
# Start API server (port 8000)
python backend/main.py
```
### 3. Frontend Setup
### 2. Start the Backend API Server
```bash
python backend/main.py
# API server running at http://127.0.0.1:8000
# Interactive API docs available at http://127.0.0.1:8000/docs
```
### 3. Start the Control Room UI
In a separate terminal:
```bash
cd frontend
npm install
npm run dev
# Access UI at http://localhost:5173
```
---
## CLI Usage Examples
## CLI Usage
The built-in Typer CLI provides terminal commands for inspection, diagnostics, and testing:
```bash
# Inspect routing decision for a coding prompt (Dry Run)
# Run system diagnostics & provider health checks
python backend/app/cli/main.py doctor
# Inspect routing decision for a prompt without executing (Dry Run)
python backend/app/cli/main.py route "Write a Python function to parse JSON"
# Route and execute a complex debugging query
python backend/app/cli/main.py run "Debug this distributed async deadlock in high-throughput worker pool"
# Route and execute a query through the selected model
python backend/app/cli/main.py run "Debug this distributed async deadlock in worker pool"
# List active models in registry
# List all registered models in the catalog
python backend/app/cli/main.py models
# View aggregate system analytics
# View system-wide routing performance and cost savings analytics
python backend/app/cli/main.py analytics
```
---
## Testing & Build Verification
## REST API Usage
### 1. Dry-Run Routing (`POST /api/route`)
```bash
curl -X POST http://127.0.0.1:8000/api/route \
-H "Content-Type: application/json" \
-d '{"prompt": "Write a quicksort algorithm in Python", "policy": "balanced"}'
```
### 2. End-to-End Routed Generation (`POST /api/generate`)
```bash
curl -X POST http://127.0.0.1:8000/api/generate \
-H "Content-Type: application/json" \
-d '{"prompt": "Explain the difference between TCP and UDP", "policy": "lowest_cost"}'
```
### 3. Fetch Registered Models (`GET /api/models`)
```bash
curl http://127.0.0.1:8000/api/models
```
---
## Running Tests
The test suite includes 15 automated unit, integration, and end-to-end tests covering prompt heuristics, candidate pruning, scoring weights, provider execution, error fallbacks, and REST endpoints:
```bash
# Run backend test suite (15 unit, integration, and e2e tests)
# Run backend test suite
pytest backend/tests
# Build frontend production bundle
# Run frontend production build test
cd frontend
npm run build
```
---
## Project Structure
```text
AI-Model-Router/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ └── feature_request.md
│ ├── pull_request_template.md
│ └── workflows/
│ └── ci.yml
├── backend/
│ ├── app/
│ │ ├── analytics/ # Cost savings and aggregate analytics service
│ │ ├── analyzer/ # Dual-mode request analyzer (heuristics & LLM)
│ │ ├── api/ # FastAPI REST endpoints and request handlers
│ │ ├── budgets/ # Spend tracking and automated threshold manager
│ │ ├── cli/ # Typer CLI application (doctor, route, run, etc.)
│ │ ├── config/ # Pydantic Settings environment configuration
│ │ ├── experiments/ # A/B policy experimentation service
│ │ ├── fallback/ # Error classifier and tiered fallback supervisor
│ │ ├── models/ # Pydantic schemas (RequestAnalysis, RoutingDecision)
│ │ ├── observability/ # Redacted structured JSON event logger
│ │ ├── providers/ # Decoupled adapters (Mock, Ollama, Cloud)
│ │ ├── router/ # Core scoring matrix, pruner, and rules engine
│ │ └── storage/ # SQLAlchemy models and SQLite async database
│ ├── tests/ # Pytest test suite (15 passing tests)
│ ├── main.py # FastAPI application entrypoint
│ └── requirements.txt # Python backend dependencies
├── frontend/
│ ├── src/
│ │ ├── components/ # UI components (Navbar, RoutingMap topology graph)
│ │ ├── pages/ # Control Room pages (Dashboard, Playground, Rules, etc.)
│ │ └── types/ # TypeScript data interfaces
│ └── package.json # Node.js dependencies
├── .env.example # Configuration template
├── .gitignore # Git exclusions
├── CODE_OF_CONDUCT.md # Contributor Covenant Code of Conduct
├── CONTRIBUTING.md # Contribution guidelines and workflow
├── LICENSE # MIT License
├── README.md # Project documentation
├── requirements.txt # Root Python dependencies
└── SECURITY.md # Vulnerability reporting and security policy
```
---
## Contributing
We welcome contributions from the community. Please review [CONTRIBUTING.md](CONTRIBUTING.md) for details on our development setup, coding standards, branch conventions, and pull request process.
Please note that this project is released with a [Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.
---
## Security
Security and privacy are core to Model Router. For vulnerability reporting procedures and our zero-secret-exposure policy, please refer to [SECURITY.md](SECURITY.md).
---
## License
MIT License. Open-source software.
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
---
## PicadoLabs
Maintained and architected by **PicadoLabs**.
- **Organization**: [PicadoLabs](https://github.com/PicadoLabs)
- **Website**: [https://picadolabs.me](https://picadolabs.me)
- **Contact**: [picadolabs@gmail.com](mailto:picadolabs@gmail.com)