fix(benchmark): fair to reasoning models, fair to router degradation, real sub-cent costs
Three false signals in benchmark results, all seen on a live run:
- Reasoning models were scored ❌ because the per-test max_tokens cap cut
them mid-<think> before the actual answer. Every run now gets +256 tokens
of headroom; non-thinking models stop at EOS as before, thinking ones get
to finish (math-mult on nanbeige: false 0/1 → honest 1/1).
- When the router degraded (timeout → fallback chain) the table showed
"✗ degraded" and counted it against classification accuracy. There was no
classification to judge — such rows now show a muted "⚠ router degraded"
with an explanatory tooltip and are excluded from the accuracy KPI.
- Costs below $0.0001 (cheap flash models) collapsed to "$0.0000". money
formatting now grows precision until two significant digits survive
($0.000035 instead of $0.0000).
2 new strings in all 7 languages.
This commit is contained in:
Vendored
+4
-1
@@ -100,13 +100,16 @@ export function confirmDialog(message) {
|
||||
export function moneyEl(n) {
|
||||
if (n == null) return document.createTextNode("—");
|
||||
if (n === 0) return document.createTextNode("$0");
|
||||
const s = n.toFixed(4);
|
||||
// sub-0.0001 costs (cheap flash models) would collapse to $0.0000 — grow
|
||||
// precision until two significant digits survive
|
||||
const s = Math.abs(n) >= 0.0001 ? n.toFixed(4) : n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
|
||||
return h("span", { class: "money" }, "$" + s.slice(0, -2), h("small", { class: "money-tail" }, s.slice(-2)));
|
||||
}
|
||||
|
||||
export function money(n) {
|
||||
if (n == null) return "—";
|
||||
if (n === 0) return "$0";
|
||||
if (Math.abs(n) > 0 && Math.abs(n) < 0.0001) return "$" + n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
|
||||
// up to 4 decimals, trailing zeros trimmed (but keep money-like 2 minimum)
|
||||
let s = n.toFixed(4).replace(/0+$/, "").replace(/\.$/, "");
|
||||
if (/\.\d$/.test(s)) s += "0";
|
||||
|
||||
Reference in New Issue
Block a user