feat: semantic (embedding-based) response cache
- cache.rs: cosine-similarity store (threshold, TTL, ring buffer), partitioned by routing signature; tracks hit rate and model cost saved - pipeline: embed routing text, look up before the model call (route_source=cache on hit), insert successful responses; stats record cache hits - config [cache] + admin settings card; dashboard shows hit-rate and savings - embedding model resolved from the pool (or first kind="embedding")
This commit is contained in:
Vendored
+16
@@ -108,6 +108,22 @@ export async function renderDashboard() {
|
||||
barList(breakdown, num)
|
||||
),
|
||||
|
||||
// semantic cache
|
||||
stats.cache && stats.cache.enabled
|
||||
? h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("dash.cache"))),
|
||||
h(
|
||||
"div",
|
||||
{ class: "kpis" },
|
||||
kpi(pct(stats.cache.hit_rate), t("dash.cache.hitRate")),
|
||||
kpi(num(stats.cache.hits), t("dash.cache.hits")),
|
||||
kpi(money(stats.cache.saved_usd), t("dash.cache.saved"))
|
||||
)
|
||||
)
|
||||
: null,
|
||||
|
||||
// recent
|
||||
h(
|
||||
"div",
|
||||
|
||||
Reference in New Issue
Block a user