Full audit of the semantic cache against a live instance (mock embedder,
identical requests): the mechanism works — miss at 3.1s, then hits at 2ms
with entries/hits/saved-$ counting. Two honest gaps fixed:
- the threshold round-tripped through f32 and rendered as
0.9200000166893005 in both the TOML and the settings input — the config
field is f64 now (writes exactly what you typed) and the input rounds
legacy values to 2 decimals;
- the card now shows proof of life (entries / hits / hit-rate / saved $)
and, when the cache is enabled with NO embedding model available, a
warning that it is silently doing nothing and how to fix it — previously
that state looked identical to a working cache.
Also audited in the same pass, no changes needed: Prometheus /metrics
(toggle on/off applies live, 404 when off), and all 7 protocol switches
(each gates its endpoint instantly both ways). 4 strings ×7 languages.
- cache.rs: cosine-similarity store (threshold, TTL, ring buffer), partitioned by
routing signature; tracks hit rate and model cost saved
- pipeline: embed routing text, look up before the model call (route_source=cache on
hit), insert successful responses; stats record cache hits
- config [cache] + admin settings card; dashboard shows hit-rate and savings
- embedding model resolved from the pool (or first kind="embedding")
infosave2007