docs: establish PicadoLabs open-source governance, CI workflows, and contributor guidelines

This commit is contained in:
Vardhman Gupta
2026-09-09 17:59:46 +05:30
parent 4e6f742272
commit 775645f600
11 changed files with 1759 additions and 47 deletions
+34
View File
@@ -0,0 +1,34 @@
---
name: Bug Report
about: Create a report to help us improve Model Router
title: "[BUG] "
labels: bug
assignees: ''
---
## Description
A clear and concise description of what the bug is.
## Environment
- OS: [e.g. Ubuntu 22.04, macOS 14.2, Windows 11]
- Python Version: [e.g. 3.11.8]
- Node.js Version: [e.g. 20.11.0]
- Router Mode: [e.g. rules, llm]
- Provider: [e.g. mock, ollama, openai]
## Steps to Reproduce
1. Start server with `...`
2. Send request to `...`
3. See error
## Expected Behavior
A clear description of what you expected to happen.
## Actual Behavior
A clear description of what actually happened.
## Logs / Screenshots
If applicable, add console output or screenshots to help explain your problem.
## Additional Context
Add any other context about the problem here.
+19
View File
@@ -0,0 +1,19 @@
---
name: Feature Request
about: Suggest an idea or capability for Model Router
title: "[FEATURE] "
labels: enhancement
assignees: ''
---
## Problem / Use Case
Is your feature request related to a problem or specific use case? Please describe.
## Proposed Solution
A clear and concise description of what you want to happen.
## Alternatives Considered
A clear and concise description of any alternative solutions or features you've considered.
## Additional Context
Add any other context or screenshots about the feature request here.
+24
View File
@@ -0,0 +1,24 @@
## Summary
Provide a brief summary of the changes introduced in this pull request.
## Motivation
Explain why this change is necessary and what issue it resolves.
## Changes Made
- List key changes and additions.
- Reference modified files and modules.
## Testing Performed
- [ ] Backend tests passing (`pytest backend/tests`)
- [ ] Frontend build passing (`cd frontend && npm run build`)
- [ ] Manual verification in Control Room or Typer CLI
## Breaking Changes
- [ ] Yes (explain below)
- [ ] No
## Checklist
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have added tests that prove my fix or feature works
- [ ] I have updated the documentation accordingly
+57
View File
@@ -0,0 +1,57 @@
name: CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
backend-test:
name: Backend Tests (Python ${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Run Pytest suite
run: |
pytest backend/tests -v
frontend-build:
name: Frontend Build & Typecheck
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Node.js 20
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
cache-dependency-path: frontend/package.json
- name: Install frontend dependencies
working-directory: frontend
run: npm ci
- name: Build and typecheck frontend
working-directory: frontend
run: npm run build
+1
View File
@@ -42,3 +42,4 @@ Thumbs.db
# Project Interview Guide & Docs
project.md
PROJECT.md
+45
View File
@@ -0,0 +1,45 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
## Scope
This Code of Conduct applies within all project spaces, and also applies when an individual is officially representing the project in public spaces.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the project team at [picadolabs@gmail.com](mailto:picadolabs@gmail.com). All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
+114
View File
@@ -0,0 +1,114 @@
# Contributing to Model Router
Thank you for your interest in contributing to Model Router. We welcome contributions from developers of all skill levels.
---
## Code of Conduct
All contributors and maintainers are expected to adhere to the [Code of Conduct](CODE_OF_CONDUCT.md). Please report unacceptable behavior to [picadolabs@gmail.com](mailto:picadolabs@gmail.com).
---
## Getting Started
### Prerequisites
- **Python**: 3.10, 3.11, or 3.12
- **Node.js**: 18+ and npm
- **Git**
- (Optional) **Ollama**: For testing local model dispatch
### Local Development Setup
1. **Fork and Clone the Repository**:
```bash
git clone https://github.com/<your-username>/AI-Model-Router.git
cd AI-Model-Router
```
2. **Set Up Python Virtual Environment**:
```bash
python -m venv venv
# On Linux/macOS:
source venv/bin/activate
# On Windows:
.\venv\Scripts\Activate.ps1
pip install -r requirements.txt
```
3. **Configure Environment Variables**:
```bash
cp .env.example .env
```
4. **Set Up Frontend**:
```bash
cd frontend
npm install
cd ..
```
5. **Verify Diagnostics**:
```bash
python backend/app/cli/main.py doctor
```
---
## Development Workflow
1. **Create a Feature Branch**:
```bash
git checkout -b feature/your-feature-name
# or for bug fixes:
git checkout -b fix/issue-description
```
2. **Make Changes**:
- Write clean, type-annotated, self-documenting code.
- Maintain the zero-secret-exposure architecture (never hardcode tokens or credentials).
- Preserve sub-3ms routing performance in the heuristics engine.
3. **Run Backend Tests**:
```bash
pytest backend/tests
```
4. **Verify Frontend Build**:
```bash
cd frontend
npm run build
cd ..
```
---
## Code Standards & Style
### Python (Backend)
- Follow PEP 8 guidelines.
- Use type hints for all function signatures and schema definitions.
- Write unit/integration tests in `backend/tests/` for new endpoints or routing logic.
- Keep external provider credentials strictly inside `.env` configuration.
### TypeScript / React (Frontend)
- Adhere to functional React component patterns with typed props.
- Use Tailwind CSS utility classes and design tokens consistent with the dark control room theme.
- Ensure `npm run build` runs with zero TypeScript or bundler errors.
---
## Pull Request Process
1. Ensure all existing 15 pytest tests pass.
2. Add tests for any new features or bug fixes.
3. Update relevant documentation in `README.md` if parameters, routes, or workflows changed.
4. Open a Pull Request on GitHub with a clear summary and motivation following the PR template.
5. A maintainer will review your submission.
---
## Questions & Support
If you have questions, open a GitHub Discussion or reach out to the maintainers at [picadolabs@gmail.com](mailto:picadolabs@gmail.com).
+2 -1
View File
@@ -1,6 +1,7 @@
MIT License
MIT License
Copyright (c) 2026 Model Router Contributors
Copyright (c) 2026 PicadoLabs
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+233 -38
View File
@@ -1,4 +1,4 @@
# Model Router
# Model Router
Intelligent, explainable, cost- and latency-aware LLM request routing platform and AI Traffic Control Room.
@@ -6,87 +6,282 @@ Intelligent, explainable, cost- and latency-aware LLM request routing platform a
## Overview
Model Router intercepts incoming AI requests, analyzes their task type and complexity, evaluates available models against a configurable multi-criteria scoring objective, selects the optimal candidate, and dispatches the request with automatic fallback handling and budget guards.
Model Router intercepts incoming AI requests, analyzes their task type and continuous complexity, evaluates available models against a configurable multi-criteria scoring objective, selects the optimal candidate, and dispatches the request with automatic fallback handling and budget guards.
The platform is designed local-first, allowing full local development and testing using Mock models or local Ollama instances without requiring paid external API keys.
---
## Core Capabilities
## Key Features
- Dual-Mode Request Analyzer: Deterministic heuristics (<3ms latency overhead) for 12 task types, continuous complexity scoring (0.0 to 1.0), and requirement detection, plus an optional LLM classifier mode.
- Explainable Routing Engine: Multi-criteria weighted scoring across Quality, Cost Efficiency, Speed, Capabilities, and Reliability with transparent decision factor reports and candidate rejection logs.
- Provider Abstraction: Decoupled adapters for Mock (simulation), Ollama (local), OpenAI, Anthropic, and Google Gemini.
- Resilience and Fallback: Automated retry classification for transient errors (timeouts, HTTP 429, 503) and tiered fallback to local/mock alternatives.
- Budget Guards: Real-time spend tracking with automated threshold interventions (80% cost optimization, 95% local-only saver, 100% block).
- Traffic Control Room UI: Real-time dark operational interface featuring live topology graphs, playground inspector, SSE live request stream, visual rules builder, and cost savings simulator.
- Developer CLI: Terminal diagnostics (`modelrouter doctor`), routing dry-run (`route`), execution (`run`), model catalog (`models`), and analytics (`analytics`).
- **Dual-Mode Request Analyzer**: Deterministic heuristics (<3ms latency overhead) for 12 task types, continuous complexity scoring (0.05 to 0.99), and requirement detection, plus an optional LLM classifier mode.
- **Explainable Routing Engine**: Multi-criteria weighted scoring across Quality, Cost Efficiency, Speed, Capabilities, and Reliability with transparent decision factor reports and candidate rejection logs.
- **Provider Abstraction**: Decoupled adapters for Mock (simulation), Ollama (local), OpenAI, Anthropic, and Google Gemini.
- **Resilience and Tiered Fallback**: Automated retry classification for transient errors (timeouts, HTTP 429, 503) and tiered fallback to local/mock alternatives.
- **Budget Control Guards**: Real-time spend tracking with automated threshold interventions (80% cost optimization, 95% local-only saver, 100% block).
- **Traffic Control Room UI**: Real-time dark operational interface featuring live topology graphs, playground inspector, SSE live request stream, visual rules builder, and cost savings simulator.
- **Developer CLI**: Terminal diagnostics (`doctor`), routing dry-run (`route`), execution (`run`), model catalog (`models`), and analytics (`analytics`).
---
## Architecture & Workflow
```text
[ Client / SDK / Typer CLI ]
│
▼
[ FastAPI Gateway (Port 8000) ]
│
┌────────┴──────────────────────────┐
│ 1. Request Analyzer (<3ms) │ --> Task Type, Complexity, Context Size
│ 2. Priority Rules Evaluation │ --> Conditional Overrides
│ 3. Candidate Hard Pruning │ --> Filter Ineligible Models (Context / Caps)
│ 4. Multi-Criteria Scoring │ --> Normalized 0-100 Score across 5 Dimensions
│ 5. Decision Factor Generator │ --> Itemized Explainability Breakdown
└────────┬──────────────────────────┘
│
▼
[ Fallback Supervisor & Provider Layer ]
├── Local: Ollama Provider (qwen2.5-coder, llama3.2, deepseek-r1)
├── Simulated: In-Memory Mock Provider (Zero Cost)
└── Cloud: OpenAI, Anthropic, Google Gemini (Optional)
│
▼
[ Storage & Observability Engine ]
├── Asynchronous SQLite WAL Database (`model_router.db`)
└── Server-Sent Events (SSE) Stream -> React Control Room (Port 5173)
```
---
## Supported Platforms & Prerequisites
### Supported Platforms
- Linux (Ubuntu 20.04+, Debian 11+, Fedora)
- macOS (macOS 12+ / Apple Silicon & Intel)
- Windows (Windows 10, Windows 11 / PowerShell & WSL2)
### Prerequisites
- Python 3.10, 3.11, or 3.12
- Node.js 18+ and npm
- (Optional) [Ollama](https://ollama.com/) for local model inference
---
## Installation & Setup
### 1. Clone the Repository
```bash
git clone https://github.com/PicadoLabs/AI-Model-Router.git
cd AI-Model-Router
```
### 2. Backend Installation
```bash
# Create and activate virtual environment
python -m venv venv
# On Linux/macOS:
source venv/bin/activate
# On Windows PowerShell:
.\venv\Scripts\Activate.ps1
# Install Python dependencies
pip install -r requirements.txt
# Create environment file from template
cp .env.example .env
```
### 3. Frontend Installation
```bash
cd frontend
npm install
cd ..
```
---
## Configuration & Environment Variables
Configuration is loaded via Pydantic Settings from the `.env` file:
| Variable | Default | Description |
| :--- | :--- | :--- |
| `APP_ENV` | `development` | Application environment (`development`, `production`, `test`) |
| `PORT` | `8000` | FastAPI server port |
| `HOST` | `0.0.0.0` | FastAPI server host |
| `DATABASE_URL` | `sqlite+aiosqlite:///./model_router.db` | SQLAlchemy database connection URI |
| `ROUTER_ANALYZER` | `rules` | Default analyzer mode (`rules` for heuristics, `llm` for model classifier) |
| `DEFAULT_ROUTING_POLICY` | `balanced` | Default routing weights (`balanced`, `lowest_cost`, `lowest_latency`, `highest_quality`) |
| `BASELINE_MODEL_ID` | `mock-power` | Reference model ID for calculating baseline cost savings |
| `DEFAULT_PROVIDER` | `mock` | Default execution provider (`mock`, `ollama`) |
| `OLLAMA_BASE_URL` | `http://localhost:11434` | Ollama HTTP endpoint |
| `OPENAI_API_KEY` | *(empty)* | Optional OpenAI API Key |
| `ANTHROPIC_API_KEY` | *(empty)* | Optional Anthropic API Key |
| `GEMINI_API_KEY` | *(empty)* | Optional Google Gemini API Key |
| `DAILY_BUDGET` | `10.00` | Daily spend limit in USD |
| `MONTHLY_BUDGET` | `100.00` | Monthly spend limit in USD |
| `MAX_RETRIES` | `2` | Maximum retries before triggering cascading fallback |
| `PROVIDER_TIMEOUT_SECONDS` | `30.0` | Provider HTTP timeout in seconds |
---
## Quickstart
### 1. Prerequisites
- Python 3.10+
- Node.js 18+ and npm
### 2. Backend Setup
### 1. Run System Diagnostics
```bash
# Clone the repository
git clone https://github.com/your-org/model-router.git
cd model-router
# Install dependencies
pip install -r requirements.txt
# Run system diagnostics
python backend/app/cli/main.py doctor
# Start API server (port 8000)
python backend/main.py
```
### 3. Frontend Setup
### 2. Start the Backend API Server
```bash
python backend/main.py
# API server running at http://127.0.0.1:8000
# Interactive API docs available at http://127.0.0.1:8000/docs
```
### 3. Start the Control Room UI
In a separate terminal:
```bash
cd frontend
npm install
npm run dev
# Access UI at http://localhost:5173
```
---
## CLI Usage Examples
## CLI Usage
The built-in Typer CLI provides terminal commands for inspection, diagnostics, and testing:
```bash
# Inspect routing decision for a coding prompt (Dry Run)
# Run system diagnostics & provider health checks
python backend/app/cli/main.py doctor
# Inspect routing decision for a prompt without executing (Dry Run)
python backend/app/cli/main.py route "Write a Python function to parse JSON"
# Route and execute a complex debugging query
python backend/app/cli/main.py run "Debug this distributed async deadlock in high-throughput worker pool"
# Route and execute a query through the selected model
python backend/app/cli/main.py run "Debug this distributed async deadlock in worker pool"
# List active models in registry
# List all registered models in the catalog
python backend/app/cli/main.py models
# View aggregate system analytics
# View system-wide routing performance and cost savings analytics
python backend/app/cli/main.py analytics
```
---
## Testing & Build Verification
## REST API Usage
### 1. Dry-Run Routing (`POST /api/route`)
```bash
curl -X POST http://127.0.0.1:8000/api/route \
-H "Content-Type: application/json" \
-d '{"prompt": "Write a quicksort algorithm in Python", "policy": "balanced"}'
```
### 2. End-to-End Routed Generation (`POST /api/generate`)
```bash
curl -X POST http://127.0.0.1:8000/api/generate \
-H "Content-Type: application/json" \
-d '{"prompt": "Explain the difference between TCP and UDP", "policy": "lowest_cost"}'
```
### 3. Fetch Registered Models (`GET /api/models`)
```bash
curl http://127.0.0.1:8000/api/models
```
---
## Running Tests
The test suite includes 15 automated unit, integration, and end-to-end tests covering prompt heuristics, candidate pruning, scoring weights, provider execution, error fallbacks, and REST endpoints:
```bash
# Run backend test suite (15 unit, integration, and e2e tests)
# Run backend test suite
pytest backend/tests
# Build frontend production bundle
# Run frontend production build test
cd frontend
npm run build
```
---
## Project Structure
```text
AI-Model-Router/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ └── feature_request.md
│ ├── pull_request_template.md
│ └── workflows/
│ └── ci.yml
├── backend/
│ ├── app/
│ │ ├── analytics/ # Cost savings and aggregate analytics service
│ │ ├── analyzer/ # Dual-mode request analyzer (heuristics & LLM)
│ │ ├── api/ # FastAPI REST endpoints and request handlers
│ │ ├── budgets/ # Spend tracking and automated threshold manager
│ │ ├── cli/ # Typer CLI application (doctor, route, run, etc.)
│ │ ├── config/ # Pydantic Settings environment configuration
│ │ ├── experiments/ # A/B policy experimentation service
│ │ ├── fallback/ # Error classifier and tiered fallback supervisor
│ │ ├── models/ # Pydantic schemas (RequestAnalysis, RoutingDecision)
│ │ ├── observability/ # Redacted structured JSON event logger
│ │ ├── providers/ # Decoupled adapters (Mock, Ollama, Cloud)
│ │ ├── router/ # Core scoring matrix, pruner, and rules engine
│ │ └── storage/ # SQLAlchemy models and SQLite async database
│ ├── tests/ # Pytest test suite (15 passing tests)
│ ├── main.py # FastAPI application entrypoint
│ └── requirements.txt # Python backend dependencies
├── frontend/
│ ├── src/
│ │ ├── components/ # UI components (Navbar, RoutingMap topology graph)
│ │ ├── pages/ # Control Room pages (Dashboard, Playground, Rules, etc.)
│ │ └── types/ # TypeScript data interfaces
│ └── package.json # Node.js dependencies
├── .env.example # Configuration template
├── .gitignore # Git exclusions
├── CODE_OF_CONDUCT.md # Contributor Covenant Code of Conduct
├── CONTRIBUTING.md # Contribution guidelines and workflow
├── LICENSE # MIT License
├── README.md # Project documentation
├── requirements.txt # Root Python dependencies
└── SECURITY.md # Vulnerability reporting and security policy
```
---
## Contributing
We welcome contributions from the community. Please review [CONTRIBUTING.md](CONTRIBUTING.md) for details on our development setup, coding standards, branch conventions, and pull request process.
Please note that this project is released with a [Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.
---
## Security
Security and privacy are core to Model Router. For vulnerability reporting procedures and our zero-secret-exposure policy, please refer to [SECURITY.md](SECURITY.md).
---
## License
MIT License. Open-source software.
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
---
## PicadoLabs
Maintained and architected by **PicadoLabs**.
- **Organization**: [PicadoLabs](https://github.com/PicadoLabs)
- **Website**: [https://picadolabs.me](https://picadolabs.me)
- **Contact**: [picadolabs@gmail.com](mailto:picadolabs@gmail.com)
+39 -8
View File
@@ -1,10 +1,41 @@
# Model Router Security Policy
# Security Policy
## Reporting Security Vulnerabilities
If you discover any security issues with API token handling, prompt injection vectors, or data exposure, please contact the maintainers immediately.
## Supported Versions
## Zero-Secret Exposure Policy
1. **API Keys**: External provider tokens (OpenAI, Anthropic, Gemini) are read strictly from local environment variables (`.env`) or secure system secrets.
2. **Logs & Transcripts**: All outgoing observability logs automatically redact authorization headers, bearer tokens, and secrets.
3. **Database Integrity**: SQLite storage only stores model identifiers and usage statistics; raw credentials are never persisted in plaintext database tables.
4. **Browser Privacy**: Frontend client queries never receive or handle backend API keys.
| Version | Supported |
| :--- | :--- |
| `1.0.x` | Yes |
| `< 1.0` | No |
---
## Reporting a Vulnerability
The Model Router team takes security and data privacy seriously. If you discover a vulnerability, please report it privately.
**DO NOT disclose vulnerabilities in public GitHub issues or discussions.**
### How to Report Privately
Please email your findings directly to the security team at:
**[picadolabs@gmail.com](mailto:picadolabs@gmail.com)**
### What to Include in Your Report
To help us triage and resolve the issue quickly, please include:
1. Description of the vulnerability.
2. Steps to reproduce the issue (including sample payloads or configuration).
3. Potential impact and attack vectors.
4. Any proposed fixes or remediations if available.
### Response Timeline
- **Initial Response**: Within 48 hours of receipt.
- **Triage & Assessment**: Within 5 business days.
- **Remediation & Patch Release**: Coordinated with the reporter prior to public release.
---
## Security Principles & Architecture
Model Router is built with defense-in-depth principles:
1. **Zero-Secret Exposure**: Provider API keys (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`) are read strictly from local environment variables and never stored in SQLite database tables or returned to web clients.
2. **Log Redaction**: Observability logging automatically sanitizes dictionary keys matching authorization headers and secret patterns.
3. **Local-First Privacy**: Complete inference can run locally via Mock or Ollama providers with zero telemetry sent to third-party cloud servers.
File diff suppressed because it is too large Load Diff