feat(benchmark): smart-routing mode is the point — live results, per-model cost
The benchmark now leads with the product's actual feature. Two explicit modes: 🧭 Smart routing (default) — every probe goes through cortiq-auto; each row shows the router's task label judged against the expected one (✓/✗ with "expected: …"), the complexity tier, the chosen model with what that request cost, and a task-rule badge when a routing rule fired. KPIs: answer quality, classification accuracy, TTFT, tok/s and money spent, plus a "requests went to" distribution with per-model cost. 🎯 Individual models — the old head-to-head comparison, now as the clearly separate option (model checkboxes appear only here). The suite covers all 8 taxonomy labels (added summarization, creative-writing and chitchat probes; expected labels annotated per test). Results are LIVE: the table grows a row after every test (with a small fade-in), the progress card shows the prompt being asked, and a 🏁 verdict toast lands at the end with the score and the total spend. 12 new strings in all 7 languages.
This commit is contained in:
Vendored
+14
@@ -579,4 +579,18 @@ export default {
|
||||
"bm.addTest": "Добавить тест",
|
||||
"bm.emptyAnswer": "пустой ответ",
|
||||
"bm.loopDetected": "зацикливание",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Умный роутинг",
|
||||
"bm.mode.models": "Отдельные модели",
|
||||
"bm.mode.routingDesc": "Проверка главной фишки продукта: каждый тест идёт через cortiq-auto — видно, какую задачу распознал роутер, какую модель выбрал, совпало ли это с ожиданием и сколько это стоило.",
|
||||
"bm.mode.modelsDesc": "Тот же набор тестов, но закреплённый за выбранными моделями — сравнение их качества, скорости и стоимости напрямую, мимо роутера.",
|
||||
"bm.answersOk": "ответы верны",
|
||||
"bm.classAcc": "классификация задач",
|
||||
"bm.modelDist": "Запросы ушли в",
|
||||
"bm.expectedLbl": "ожидалось:",
|
||||
"bm.answer": "Ответ",
|
||||
"bm.liveRunning": "идёт…",
|
||||
"bm.spent": "потрачено",
|
||||
"bm.verdict": "🏁 Готово: ответы {p}, потрачено {s}",
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user