docs: add latency benchmark vs LiteLLM/Portkey

- bench/ reproducible harness: instant mock backend + Apache Bench (ab -k -r)
- measures gateway overhead in passthrough mode (apples-to-apples)
- BENCHMARKS.md: methodology, caveats, results (cortiq ~57k rps p99 1ms vs
  Portkey ~5.8k p99 9ms vs LiteLLM 4-workers ~1.2k p99 59ms)
- README (EN+RU): Performance section + highlight

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
infosave2007
2026-06-30 18:11:26 +03:00
co-authored by Claude Opus 4.8
parent e015555dff
commit a5966dd479
8 changed files with 279 additions and 1 deletions
+1
View File
@@ -0,0 +1 @@
{"model":"bench","messages":[{"role":"user","content":"ping"}],"max_tokens":1}