fix(benchmark): fair to reasoning models, fair to router degradation, real sub-cent costs
Three false signals in benchmark results, all seen on a live run:
- Reasoning models were scored ❌ because the per-test max_tokens cap cut
them mid-<think> before the actual answer. Every run now gets +256 tokens
of headroom; non-thinking models stop at EOS as before, thinking ones get
to finish (math-mult on nanbeige: false 0/1 → honest 1/1).
- When the router degraded (timeout → fallback chain) the table showed
"✗ degraded" and counted it against classification accuracy. There was no
classification to judge — such rows now show a muted "⚠ router degraded"
with an explanatory tooltip and are excluded from the accuracy KPI.
- Costs below $0.0001 (cheap flash models) collapsed to "$0.0000". money
formatting now grows precision until two significant digits survive
($0.000035 instead of $0.0000).
2 new strings in all 7 languages.
This commit is contained in:
Vendored
+2
@@ -616,4 +616,6 @@ export default {
|
||||
"keys.h.st.installed": "Installé sur la machine de la passerelle",
|
||||
"keys.h.st.configured": "Déjà configuré pour cette passerelle",
|
||||
"keys.h.st.notFound": "Introuvable sur la machine de la passerelle",
|
||||
"bm.degraded": "routeur dégradé",
|
||||
"bm.routerDegraded": "Le routeur n'a pas répondu à temps — la requête a suivi la chaîne de secours ; la classification n'est pas comptée.",
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user