infosave2007
6466006de6
- bench/ reproducible harness: instant mock backend + Apache Bench (ab -k -r) - measures gateway overhead in passthrough mode (apples-to-apples) - BENCHMARKS.md: methodology, caveats, results (cortiq ~57k rps p99 1ms vs Portkey ~5.8k p99 9ms vs LiteLLM 4-workers ~1.2k p99 59ms) - README (EN+RU): Performance section + highlight
2 lines
79 B
JSON
2 lines
79 B
JSON
{"model":"bench","messages":[{"role":"user","content":"ping"}],"max_tokens":1}
|