feat: semantic (embedding-based) response cache
- cache.rs: cosine-similarity store (threshold, TTL, ring buffer), partitioned by routing signature; tracks hit rate and model cost saved - pipeline: embed routing text, look up before the model call (route_source=cache on hit), insert successful responses; stats record cache hits - config [cache] + admin settings card; dashboard shows hit-rate and savings - embedding model resolved from the pool (or first kind="embedding")
This commit is contained in:
@@ -9,6 +9,7 @@
|
||||
#![allow(dead_code)]
|
||||
|
||||
mod admin;
|
||||
mod cache;
|
||||
mod config;
|
||||
mod error;
|
||||
mod model;
|
||||
|
||||
Reference in New Issue
Block a user