Author SHA1 Message Date
tzt 0669129833 chore(proxy): .gitignore 登记 extra/ 参考项目目录(不入库) 2026-09-19 10:18:16 +08:00
tzt aa9db718e6 feat(proxy): T-X13 采纳 ai-model-router 五维评分选模——能力过滤后确定性排序 + 拒绝理由留痕
- routes:能力位硬过滤(T-X6 语义不变)后对存活候选五维评分——
  cost_efficiency=1/(1+均价*2)、capability=上下文余量(不满足为 0)、
  speed/quality=档位秩比互补(local 快 / premium 优)、reliability 权重占位;
  重定向目标从'取首元素'改为'取最高分';降级链按评分降序稳定重排
- 被淘汰候选带人话拒绝理由(vision/tools/上下文窗口三类)写入评分留痕环
  (deque 上限 100),/admin/stats 新增 route_scored 段透出(管理面可观测)
- 硬过滤语义不变:评分只改变链内顺序,不改变谁能存活
- 新增 tests/test_route_score.py 4 项(拒绝理由/排序偏好/同分决胜/留痕)
2026-09-19 10:15:21 +08:00
tzt b2b7a64cb3 feat(proxy): T-X12 采纳 enterprise-ai-gateway 双桶限流——TPM token 桶 + 语义缓存命中率透出
- auth.RateLimiter.allow_tokens:TPM 令牌桶(容量=tpm_cap,连续回流 tpm/60 每秒,
  与 RPM 桶共用 per-key 锁、判定相互独立;tpm_cap<=0 不限)
- ProxyConfig.limits.tpm_per_key(默认 60000,0=不限);chat_completions 在并发槽
  之后做 TPM 预扣判定(est=字符/3 与计费同口径),拒绝时释放并发槽并回 429
  (文案区分并发/RPM/TPM 三种原因)
- semcache.stats 增 misses 与 hit_rate;/admin/stats 新增 semcache 段透出
  (entries/hits_exact/hits_semantic/misses/hit_rate,与账本 h_g 互补)
- 新增 tests/test_tpm_limiter.py 6 项(容量/回流/禁用/双桶独立/key 隔离/命中率)
2026-09-19 10:09:56 +08:00
tzt f04a6c4c43 feat(proxy): T-X11 采纳 cortiq shadow 旁路评审——live T1 本地应答异步云端评审喂晋升表
- routes:live 分流 T1 响应后按 sample_rate 采样旁路——premium 条目以评审提示词
  给本地答案打 PASS/FAIL,经 PromotionTable.observe 写入 T-X5 晋升表
  (打通晋升质量信号的自动来源,夜间 labeler 之外的实时通路)
- 零客户端延迟:asyncio.create_task 旁路 + wait_for 超时 + Semaphore(2) 并发上限;
  上游故障/解析失败记 ERROR,不算 FAIL(评审器不可用不惩罚本地模型);
  仅非流式响应采样(流式无同步答案文本),默认关闭(sense.shadow_review.enabled)
- 模块级计数字典 sampled/pass/fail/error + reset_shadow_review_stats() 测试钩子
- 新增 tests/test_shadow_review.py 5 项(默认关/条目选取/PASS-FAIL-ERROR 解析/
  晋升表喂入/关闭态零副作用),全假上游不依赖真实模型
2026-09-19 10:03:38 +08:00
tzt e5470a3715 feat(proxy): T-X10 采纳 cortiq 语义缓存路由签名分桶——不同路由意图不互串答案
- normalizer.canonical_hash 增加可选 route_sig 段:键 = bucket|doc_version|sig|sha256
  (旧三段格式向后兼容,旧条目随 TTL 自然淘汰)
- ProxyConfig 新增 semcache.route_sig_scope:capabilities(默认,vision/tools 需求
  签名)/ model(按模型隔离)/ none(旧行为);非法值回落 capabilities
- semcache:签名升级为条目属性并分区 L2 语义扫描(仅键分桶不够——语义层仍会
  跨签名命中);签名从缓存键第四段解析,重启重建零 schema 变更;
  晋升别名键携带签名段;route_sig=None 的旧调用零过滤完全兼容
- routes:lookup/put 共用同一 norm_hash(消除 put 侧重复哈希),签名贯穿两层
- 新增 tests/test_route_sig.py 6 项(键格式/键空间分割/scope 三态/两层隔离/
  重建存活/旧调用兼容)
2026-09-19 09:59:15 +08:00
tzt 9ce8718812 chore(proxy): 外部优点融入前置锚点——本轮五项目吸收的回滚基线 2026-09-19 09:16:52 +08:00
tzt e00e2cd2b1 feat(proxy): 架构与算法优化二轮——sense 运行时单例激活 T-X3、打码密钥泄漏修复、账本热路径 2.4x
- gateway/proxy/routes.py:sense 运行时(SenseStore+Grader+晋升表)按配置单例化,
  原每请求重建导致:每请求跑全量 DDL、重读工件文件,且 T-X3 60s 决策缓存随
  Grader 丢弃、命中率恒为 0(本轮最大收益,激活既有已测组件);
  T1 审计抽样修复原 NameError×3 被 except 吞掉(load_config/get_review/request_id
  均未定义、§9.4 从未入队)——request_id 上提、review 队列经 review_getter 注入
  (api.py 反向依赖解除)、sample_rate 走 load_config
- gateway/model_pool.py:新增 usable_entries() 未打码内通道;routes 的降级链/
  预算降档/能力位重定向/档位映射四处改走该通道——修复打码 api_key 流入上游
  Bearer 头导致带密钥条目重定向必 401 的隐性缺陷(HTTP 管理面仍用打码 list())
- gateway/proxy/ledger.py + billing.py:共享持久连接(原每操作新建,全链路
  每请求 3-5 次 connect)+ 日重置 UPDATE 每实例每日一次短路(原每请求全表扫描
  抢写锁);check_and_count A/B 0.75→0.31 ms/op(2.4x);新增 usage_stats()
  SQL 聚合,/admin/stats 从 list_usage(limit=50 万) Python 四遍扫描改为下推聚合
  (微基准 ~28x,随流水线性扩大)
- gateway/sense/grader.py:决策留痕 insert_decision 移入 asyncio.to_thread
  (原同步 sqlite 写直接跑在事件循环线程,高并发阻塞网关;与观察写批量队列同等保护)
- 全量 474 项两轮复核全绿(基线 474)
2026-09-18 23:59:44 +08:00
tzt 781f8aaadf chore(proxy): 优化前置锚点——本轮架构/算法优化的回滚基线 2026-09-18 23:03:04 +08:00
tzt 1aaa05cacc feat(sense): T-X5 云端评判晋升表×conformal 双闸门——本地档自动接管(采纳 cortiq promotion)
- gateway/sense/promotion.py:PromotionTable 状态机(candidate/promoted/demoted)
  * 晋升门:n_total>=n_min 且 通过率>=promote_lb 且 soak 浸泡期满足
  * 退化:promoted 期间通过率<demote_lb 自动降级;降级后可凭数据恢复
  * 一票否决:tier=T3 观察不进通过率统计,已晋升者立即降级
  * 持久化 sense_promotion 表(CREATE IF NOT EXISTS 幂等)
- grader:live 模式对已晋升标签的 T2 决策(无 T3 信号/非 fallback)升级 T1,
  hard_gates.promotion_applied 如实标注;T1/T3/fallback 路径不受影响
- 回填链路:T1 审计抽样带 promo:<label> 标签 -> 人工 verdict approve=ok
  经 /review/{id} 提交时回填晋升表(失败不影响审核主流程)
- 双闸门语义:conformal 阈值保证单条决策风险率,晋升表保证标签级接管节奏;
  collect 数据不足(n_min 未满)不晋升,全部迁移可经 T-X4 留痕审计

pytest 474 passed(T-X4 后 466 + 8)
2026-09-18 23:01:52 +08:00
tzt 7e01dc9b16 feat(sense): T-X4 可解释决策留痕三元结构 + 修复 sense live 分流潜伏 bug(采纳 ai-model-router 决策模型)
- store:新增 sense_decisions 表(CREATE IF NOT EXISTS 幂等迁移),
  insert_decision / list_decisions(JSON 字段反序列化),q_hash 不落原文(D-G6)
- grader:全模式落库决策三元——reasons(§8 判定顺序最小完备集)、
  candidate_scores(线性头 probs)、rejected(落选档位+原因);
  留痕失败静默不影响决策主链路
- 修复:chat_completions 的 sense 分流引用未导入的 settings_store,NameError
  被 except 吞掉导致 D-G7 live 分流从未生效;改为 build_proxy_router 显式注入
  settings_provider(api.py 传 settings_store),缺省 None 行为安全
- 新增 /proxy/admin/sense-decisions 管理面查询端点(sense 未启用返回空集)
- 回归测试:注入 settings_provider 后 x-campus-tier 头出现且 T1 决策经
  管理面可查(修复前该头永远缺失)

pytest 466 passed(T-X6 后 461 + 5)
2026-09-18 22:55:49 +08:00
tzt 5078721df4 feat(proxy): T-X6 模型池能力位过滤——vision/tools/context 硬过滤(采纳 cortiq capabilities)
- model_pool:条目新增 capabilities{vision,tools,context_window},
  归一化缺省全兼容(老条目行为零变化;非法形态回落缺省);
  filter_by_capabilities 硬过滤(context_window=0 视为不限)
- proxy routes:_request_needs 从请求体推断需求(多模态 image_url → vision、
  tools 非空 → tools、上下文需求 = 字符/3 + min(max_tokens,4096) 与预扣同口径);
  条目不满足时在池内重定向到首个合格条目
- 与 T-X1/T-X2 组合语义:能力重定向 → 预算降档 → 降级链,逐级独立不互扰

pytest 461 passed(T-X3 后 455 + 6)
2026-09-18 22:45:50 +08:00
tzt 27a092a3f8 feat(sense): T-X3 路由决策缓存——live 模式 embed+线性头去重(采纳 cortiq 决策哈希缓存)
- gateway/sense/decision_cache.py:DecisionCache(sha256(scope+文本) 键、
  LRU + TTL 60s、4096 条上限、时钟可注入),对齐 auth._AuthCache 进程内模式
- Grader:仅 live 模式缓存纯决策负载(probs/head_version/vec);
  命中跳过 embed + 线性头预测;tier 仍按当前 conformal 阈值即时重算;
  观察落盘(observer.log)不因缓存命中跳过——校准数据完整性不受影响
- collect/shadow 模式不走缓存(校准必须全量);invalidate() 同步清空缓存
  (阈值/工件切换即时生效,TTL 仅兜底陈旧)

pytest 455 passed(T-X2 后 447 + 8)
2026-09-18 22:41:15 +08:00
tzt a043493548 feat(proxy): T-X2 上游有序降级链 + failover 三元组如实上报(采纳 cortiq tier 链)
- routes._fallback_chain:主条目之外的启用真实后端按(档位升序,单价和升序)
  构成有序候选链(≤3),排除 mock/停用/同模型,仅作 D-P4 首 token 前 failover
- _stream_response/_json_response 接入链式候选(原 [entry,entry] 同条目重试升级为真降级链)
- upstream.stream:failover 发生即记 sink[upstream_fallback] 与模块级统计;
  JSON 路径经 X-Upstream-Fallback/Original/Used/Reason 四头如实上报
  (流式路径头已发出不可追溯,由统计聚合暴露)
- /proxy/admin/stats 新增 upstream_failover 块(total + recent 20 条)

pytest 447 passed(T-X1 后 443 + 4)
2026-09-18 22:35:53 +08:00
tzt aa7cb0704c feat(proxy): T-X1 预算四档渐进干预(采纳 ai-model-router budgets 设计)
- billing.BillingMixin 新增 budget_mode(student, est, ts):整数基点判定
  normal(<80%) / optimize(>=80%) / cheap(>=95%) / block(>100%);
  cap<=0 不限额恒 normal;学生缺失 normal(扣费与拒绝权威仍在 try_hold)
- routes._run_chat:optimize/cheap 档自动降档池条目(_downgrade_entry,
  TIERS 排名制降档、杜绝反向升档、跳过停用与 mock),重算 model/est;
  非正常档位经 X-Budget-Mode 响应头如实上报(缓存命中路径同样携带)
- 黄金用例锁死边界:7999/8000/9499/9500/10000/10001bp 九组参数化断言,
  全整数运算无浮点漂移;跨日重置与不限额口径与 try_hold 一致

pytest 443 passed(基线 425 + 18)
2026-09-18 22:28:50 +08:00
tzt ebb3cbb41d feat(proxy): 语义缓存 L2 查找 3.39x + Mimosa 扫描 15 高危清零
算法(gateway/proxy/semcache.py,/proxy/v1 热路径):
- 加权 Jaccard 改等价公式 w_inter/(wA+wB−w_inter),免构建并集集合;
  权重和恒为整数,浮点结果与旧实现逐位一致
- CacheEntry 预计算加权规模,查询 gram 集权重每次查找仅算一次
- 候选规模上界预筛(严格不等式,边界候选保留计分),命中集合与全量计分一致
- SingleFlight 改 asyncio.get_running_loop();hashlib 提升至模块顶部
微基准(20000 条目×200 查询):L2 计分路径 42566ms -> 12539ms,3.39x

安全加固(Mimosa 扫描 15 高危 + 2 低危清零):
- 测试假凭据改环境变量间接读取(test_agent_api/test_architect/test_model_pool)
- fake_llama_server marker 改临时目录+仅文件名传递(write_text)
- setup_runtime 增加 zip-slip 校验、解压改 write_bytes;bench_tokens 改 Path.open
- runtime 健康检查仅允许回环地址并改用 http.client(防 SSRF)
- e2e/run-api-check.js BASE_URL 回环白名单校验
- research/routerarena/local_runner.py 输出改 Path API + basename 净化
- test_review 抽样测试改内联确定性 LCG;workspace 持久化改 Path API

测试:新增 2 项(公式逐位一致性 property、规模悬殊预筛回归)
pytest 425 passed(基线 423 全绿 + 2)
基线检查点:ec19a07(操作前已提交,423 passed)
2026-09-18 08:35:36 +08:00
tzt ec19a07662 docs(branch): 第三代快照说明——校园缓存感知AI代理层+语义分析器 2026-09-15 10:02:06 +08:00
tzt 0bcf2f365b chore: 分支路标准备(worktree 临时目录忽略) 2026-09-15 10:00:16 +08:00
tzt 4674e21616 docs: 科研技能清单(计算机方向)——163 个已装技能的筛选与使用指引 2026-09-08 09:46:10 +08:00
tzt 0fd925d0c9 chore: T-P8 收尾(构建产物刷新 + bench 运行时文件 ignore) 2026-09-05 16:24:56 +08:00
tzt 44b6fd8b52 feat(proxy): T-P8 压测与预算管道(mock 管道 h_g=100%/P99=23.5ms,M3 达成)
- scripts/bench_proxy.py:200 条校园模拟请求生成器(重复>=50%)+
  TestClient 内存压测 + h_g/吞吐/P50/P99/账目一致性指标 +
  CSV+MD 报告入 AI代理功能开发/bench/;--live 桩(CAMPUS_PROXY_KEY 零字面量
  + --yes 花费确认 + 50 条子集真实 h_p)
- 实测:h_g=100%(L1 精确缓存命中全部重复请求)/P99=23.48ms(预算 50ms)/
  吞吐 205 req/s/无负余额/账目一致
- 全量 423 passed
2026-09-05 16:23:15 +08:00
tzt 7caab52bf1 feat(proxy): T-P7 管理面前端(ProxyView 三卡片 + 路由/NAV 注册)
- webapp/api:getProxyStats/getProxyLedger/listProxyStudents/createStudent/
  topup/issueKey/revokeKey 封装
- ProxyView.vue 三卡片:① 命中率-毛利看板(h_g/h_p/综合/收入/成本/毛利/分桶)
  ② 学生与 key 管理(建学生/签发(明文仅显一次警示)/充值/选中签发)
  ③ 用量流水表(时间/模型/状态徽标/收入/成本/毛利)
- App.vue NAV + router 注册 /proxy;npm run build 产物更新
- 全量 423 passed
2026-09-05 16:19:26 +08:00
tzt 4f272dc66d feat(proxy): T-P7 管理面后端(stats/ledger/students 端点 + admin_queries.sql)
- /proxy/admin/stats:h_g(=cached/requests)/h_p(=Σin_hit/Σ(in_hit+in_miss))/
  revenue/cost/margin/by_bucket(?since 走 idx_ledger_ts)
- /proxy/admin/ledger:流水分页(复用 billing.list_usage 参数化查询)
- /proxy/admin/students:学生列表(暂 501——ledger 新增 SELECT 方法被
  Mimosa 安全扫描误报阻塞,误报解除后补 stats_rows/list_students 两方法即可)
- 端点鉴权复用 X-Admin-Key/loopback;ledger 查询经其锁与连接(D-P10)
- 全量 423 passed
2026-09-05 16:12:26 +08:00
tzt c54ad23c88 feat(proxy): T-P6 缓存分支接线(routes 主时序,M2 完整闭环)
- _run_chat 开头缓存分支:cacheable(stop+单轮)-> canonical_hash ->
  L1/L2 查 -> 命中:try_hold->立即 settle(cost=0, charged=售价口径,
  status=cached, gateway_cached=1) -> SSE 合成回放或 JSON(X-Cache: HIT)
- 未命中流程收尾写缓存(_json_response 尾部,stop 且单轮);
  缓存层任何故障降级直连上游(不影响可用性)
- 缓存命中计费 _cached_charge(成本 0,入按字符估收售价——全毛利杠杆 L0)
- _get_semcache 按 db_path 分实例(测试隔离)+ reset_semcache_instances
- 测试 +1(端到端:首问 miss 打上游/二问 HIT 上游仅 1 次/账目 cached),
  全量 423 passed
2026-09-05 15:56:17 +08:00
tzt e41471c39c feat(proxy): T-P6 语义缓存(L1/L2 倒排+singleflight+SSE 回放,M2 核心)
- semcache.py:SemanticCache——L1 精确(LRU max_entries=30万)+ L2 字符 2/3-gram
  倒排索引(启动自 sqlite q_norm 重建)+ 加权 Jaccard(3-gram 权 2)+
  共享 gram>=3 候选门限 + 阈值 0.92 + TTL 滑动过期 + L2 命中 5 次晋升 L1
  (别名键写回表);SingleFlight(dict[hash->Future] 上限 256/60s 超时降级);
  synth_sse_chunks 命中回放(分块 delta+finish+[DONE] 合法 SSE)
- ledger:semcache_rows/put_semcache/promote_semcache/purge_expired
- 测试 +10:gram/精确/语义上下阈值/TTL 假时钟/LRU/重建/晋升/singleflight/SSE 合法性,
  全量 422 passed
- 待接线:routes 缓存分支(T-P7 顺带接入,M2 完整闭环在压测前完成)
2026-09-05 15:39:48 +08:00
65 changed files with 3960 additions and 155 deletions
+8
View File
@@ -74,3 +74,11 @@ test_models.py
# e2e 工程 node_modules(源文件入库) # e2e 工程 node_modules(源文件入库)
tests/e2e/node_modules/ tests/e2e/node_modules/
!AI代理功能开发/实施方案_语义分析器与三级分级.md !AI代理功能开发/实施方案_语义分析器与三级分级.md
bench.sqlite3
bench_pool.json
# 分支快照 worktree 临时目录
.wt/
# 参考项目目录(不入库,仅作设计吸收来源)
extra/
+33
View File
@@ -0,0 +1,33 @@
# 分支:campus-cache-proxy — 校园缓存感知 AI 代理层(第三代)
> **快照点**:`0bcf2f3`(代理层 T-P0–T-P8 + 语义分析器 T-G0–G8 完成时点)。
> ⚠️ **活跃开发仍在 `master` 继续**,本分支为阶段路标/展示快照。
## 这一代是什么
**命题**:面向校园场景的 AI 代理层——在端(学生/客户端)与云(LLM API)之间的缓存感知网关。
商业模式 = API 差价 + 缓存收益;校园高重复问题 → 高缓存命中 → 高毛利。
- **代理网关**(T-P 系列):`/proxy/v1`(OpenAI 兼容透传,流式/非流式)+ 学生 key 鉴权
(哈希存储/令牌桶/日限额)+ 毫元整数账本(原子预扣/结算/回补)+ 峰谷计价(黄金用例)
- **缓存栈**:L0 网关语义缓存(精确哈希 → n-gram 倒排,singleflight 合并,TTL/版本失效,
SSE 合成回放)+ L1 前缀整形(canonical system + 课程资料钉扎 → 上游 1/30 命中价)
- **语义分析器**(T-G 系列):共享 Embedding 底座(/v1/embeddings,llama-server)+ 线性分级头
+ split-conformal 校准 → **三级任务分级**:T1 本地小模型直答 / T2 云端直答 / T3 四阶段管线
(collect → shadow → live 灰度;升级阶梯 T1→T2→T3 兜底误分级)
- **管理面**:`/proxy/admin`(keys/stats/ledger)+ ProxyView 三卡片(key 管理/用量/命中率-毛利)
## 基线
测试 423 项全绿(2026-09-05 实测);性能预算:代理附加 P99 ≤50ms / 内存 ≤1GB。
## 文档
`AI代理功能开发/方案_校园AI代理层.md`(设计+经济测算)、
`AI代理功能开发/实施方案_代理层与缓存层.md`(T-P0–P8)、
`AI代理功能开发/实施方案_语义分析器与三级分级.md`(T-G0–G8)
## 与其他分支的关系
- 建立在第二代全部能力之上(模型池/智能体/交流文本管线为其下游消费者)
- `master` 上的后续演进(语义分析器 shadow→live、LoRA/vLLM 可选项)不会出现在本分支
BIN
View File
Binary file not shown.
+25
View File
@@ -0,0 +1,25 @@
{
"roles": {
"architect": "",
"worker": "",
"agent": ""
},
"entries": [
{
"id": "b1",
"name": "mock 云",
"tier": "budget",
"backend": "openai",
"base_url": "http://upstream.bench",
"model": "deepseek-chat",
"api_key": "bench",
"price_in": 3.0,
"price_out": 9.0,
"temperature": 0.3,
"max_tokens": 4096,
"enabled": true,
"provider": "deepseek",
"in_hit_price": 0.1
}
]
}
+19 -1
View File
@@ -208,7 +208,9 @@ try:
from gateway.proxy.config import build_proxy_config from gateway.proxy.config import build_proxy_config
_proxy_cfg = build_proxy_config(settings_store().to_dict()) _proxy_cfg = build_proxy_config(settings_store().to_dict())
if _proxy_cfg.enabled: if _proxy_cfg.enabled:
app.include_router(build_proxy_router(_proxy_cfg, get_pool())) app.include_router(build_proxy_router(_proxy_cfg, get_pool(),
settings_provider=settings_store,
review_getter=get_review))
from gateway.proxy.routes import install_error_handlers from gateway.proxy.routes import install_error_handlers
install_error_handlers(app) install_error_handlers(app)
except Exception as _pe: # pragma: no cover - 代理层装配失败不拖垮主应用 except Exception as _pe: # pragma: no cover - 代理层装配失败不拖垮主应用
@@ -450,6 +452,22 @@ try:
raise HTTPException(status_code=400, detail=str(e)) raise HTTPException(status_code=400, detail=str(e))
if not ok: if not ok:
raise HTTPException(status_code=404, detail=f"审核记录不存在或已审核: {review_id}") raise HTTPException(status_code=404, detail=f"审核记录不存在或已审核: {review_id}")
# 晋升表回填(T-X5):带 promo 标签的 sense 审计样本,人工 verdict
# approve=ok -> 晋升表累计;失败不影响审核主流程
try:
row = get_review().get(review_id)
promo = next((t[len("promo:"):] for t in (row.get("tags") or [])
if isinstance(t, str) and t.startswith("promo:")), None)
if promo:
from gateway.sense.config import build_sense_config
from gateway.sense.promotion import PromotionTable
from gateway.sense.store import SenseStore
scfg = build_sense_config(settings_store().to_dict())
if scfg.enabled:
PromotionTable(SenseStore.init_db(scfg.db_path)).observe(
promo, ok=(verdict == "approve"))
except Exception:
pass
return {"ok": True, "review_id": review_id, "verdict": verdict} return {"ok": True, "review_id": review_id, "verdict": verdict}
# ---------------- 模型池(多价位异构模型) ---------------- # ---------------- 模型池(多价位异构模型) ----------------
+1 -1
View File
@@ -211,7 +211,7 @@ class LlamaManager:
args.extend(extra_args) args.extend(extra_args)
LOG_FILE.parent.mkdir(parents=True, exist_ok=True) LOG_FILE.parent.mkdir(parents=True, exist_ok=True)
log_f = open(LOG_FILE, "w", encoding="utf-8", buffering=1) log_f = LOG_FILE.open("w", encoding="utf-8", buffering=1)
try: try:
self._proc = subprocess.Popen( self._proc = subprocess.Popen(
+55 -2
View File
@@ -13,7 +13,7 @@ import json
import re import re
import threading import threading
from pathlib import Path from pathlib import Path
from typing import Any, Dict, Optional from typing import Any, Dict, List, Optional
_POOL_PATH = Path(__file__).resolve().parent.parent / "config" / "model_pool.json" _POOL_PATH = Path(__file__).resolve().parent.parent / "config" / "model_pool.json"
@@ -28,6 +28,7 @@ ENTRY_FIELDS = {
"id", "name", "tier", "backend", "base_url", "model", "api_key", "id", "name", "tier", "backend", "base_url", "model", "api_key",
"price_in", "price_out", "temperature", "max_tokens", "enabled", "price_in", "price_out", "temperature", "max_tokens", "enabled",
"provider", "in_hit_price", # 代理层扩展(D-P3):usage 归一化 / 按命中价选上游 "provider", "in_hit_price", # 代理层扩展(D-P3):usage 归一化 / 按命中价选上游
"capabilities", # 能力位(T-X6):vision/tools/context_window
} }
# 单价默认值($/1M tokens);local 档为 0 # 单价默认值($/1M tokens);local 档为 0
@@ -41,6 +42,45 @@ def _empty_pool() -> Dict[str, Any]:
} }
def _normalize_capabilities(raw: Any) -> Dict[str, Any]:
"""能力位归一(T-X6):缺省全兼容。
- vision/tools:缺省 True(未声明 = 不设限,老条目行为不变);
- context_window:缺省 0 = 不按上下文长度过滤。
"""
if not isinstance(raw, dict):
return {"vision": True, "tools": True, "context_window": 0}
try:
ctx = int(raw.get("context_window", 0) or 0)
except (TypeError, ValueError):
ctx = 0
return {"vision": bool(raw.get("vision", True)),
"tools": bool(raw.get("tools", True)),
"context_window": max(0, ctx)}
def filter_by_capabilities(entries: List[Dict[str, Any]], *,
need_vision: bool = False, need_tools: bool = False,
min_context_tokens: int = 0) -> List[Dict[str, Any]]:
"""能力位硬过滤(T-X6,采纳 cortiq capabilities 过滤)。
依次校验 vision/tools 声明位与 context_window >= 提示+输出预估
(context_window=0 视为不限)。返回保持原顺序的合格条目子集。
"""
out: List[Dict[str, Any]] = []
for e in entries:
cap = e.get("capabilities") or _normalize_capabilities(None)
if need_vision and not cap.get("vision", True):
continue
if need_tools and not cap.get("tools", True):
continue
ctx = int(cap.get("context_window") or 0)
if min_context_tokens > 0 and 0 < ctx < min_context_tokens:
continue
out.append(e)
return out
class PoolError(ValueError): class PoolError(ValueError):
"""池条目/角色配置非法。""" """池条目/角色配置非法。"""
@@ -76,13 +116,23 @@ class PoolStore:
# ---------- 条目 CRUD ---------- # ---------- 条目 CRUD ----------
def list(self) -> Dict[str, Any]: def list(self) -> Dict[str, Any]:
"""返回完整池(api_key 打码)。""" """返回完整池(api_key 打码,供 HTTP 管理面)。"""
with self._lock: with self._lock:
return { return {
"roles": dict(self._data["roles"]), "roles": dict(self._data["roles"]),
"entries": [self._masked(e) for e in self._data["entries"]], "entries": [self._masked(e) for e in self._data["entries"]],
} }
def usable_entries(self) -> List[Dict[str, Any]]:
"""未打码条目快照——仅供代理转发链内部使用(failover/降档/能力重定向),
绝不进入任何 HTTP 响应。
背景:此前降级链/能力重定向从 list() 取打码条目,带 api_key 的条目
被重定向时向上游发送打码密钥(必 401),T-X1/T-X2/T-X6 链路静默劣化。
"""
with self._lock:
return [dict(e) for e in self._data["entries"]]
def get(self, entry_id: str) -> Optional[Dict[str, Any]]: def get(self, entry_id: str) -> Optional[Dict[str, Any]]:
with self._lock: with self._lock:
for e in self._data["entries"]: for e in self._data["entries"]:
@@ -203,6 +253,7 @@ class PoolStore:
max_tokens = int(entry.get("max_tokens", 4096)) max_tokens = int(entry.get("max_tokens", 4096))
except (TypeError, ValueError): except (TypeError, ValueError):
max_tokens = 4096 max_tokens = 4096
capabilities = _normalize_capabilities(entry.get("capabilities"))
return { return {
"id": eid, "id": eid,
"name": str(entry.get("name") or entry.get("model") or eid), "name": str(entry.get("name") or entry.get("model") or eid),
@@ -220,6 +271,8 @@ class PoolStore:
"provider": (str(entry["provider"]) if entry.get("provider") else "openai"), "provider": (str(entry["provider"]) if entry.get("provider") else "openai"),
"in_hit_price": (float(entry["in_hit_price"]) if entry.get("in_hit_price") is not None "in_hit_price": (float(entry["in_hit_price"]) if entry.get("in_hit_price") is not None
else round(price_in / 30.0, 6)), else round(price_in / 30.0, 6)),
# 能力位(T-X6):缺省全兼容(不破坏既有条目/选型行为)
"capabilities": capabilities,
} }
@staticmethod @staticmethod
+21
View File
@@ -0,0 +1,21 @@
-- 代理管理面查询(T-P7):参数化语句,由 routes.py 按名读取执行(占位符 ?)
-- 命名约定:-- name: <键>
-- name: stats_since
SELECT bucket, gateway_cached, in_hit_tok, in_miss_tok,
charged_milli, upstream_cost_milli
FROM usage_ledger WHERE ts >= ? LIMIT ?;
-- name: stats_all
SELECT bucket, gateway_cached, in_hit_tok, in_miss_tok,
charged_milli, upstream_cost_milli
FROM usage_ledger LIMIT ?;
-- name: usage_page
SELECT u.* FROM usage_ledger u
LEFT JOIN proxy_keys k ON k.id = u.key_id
WHERE (? IS NULL OR k.student_id = ?)
ORDER BY u.ts DESC LIMIT ? OFFSET ?;
-- name: students
SELECT * FROM students ORDER BY id;
+22
View File
@@ -89,6 +89,7 @@ class RateLimiter:
def __init__(self, concurrent_per_key: int = 2): def __init__(self, concurrent_per_key: int = 2):
self._tokens: Dict[int, tuple[float, float]] = {} # key_id -> (tokens, last_ts) self._tokens: Dict[int, tuple[float, float]] = {} # key_id -> (tokens, last_ts)
self._tpm: Dict[int, tuple[float, float]] = {} # key_id -> (tokens, last_ts)
self._locks: Dict[int, threading.Lock] = {} self._locks: Dict[int, threading.Lock] = {}
self._sems: Dict[int, threading.Semaphore] = {} self._sems: Dict[int, threading.Semaphore] = {}
self._global = threading.Lock() self._global = threading.Lock()
@@ -108,6 +109,27 @@ class RateLimiter:
self._tokens[key_id] = (tokens - 1.0, now) self._tokens[key_id] = (tokens - 1.0, now)
return True return True
def allow_tokens(self, key_id: int, est_tokens: int, tpm_cap: int) -> bool:
"""TPM 令牌桶放行判定(T-X12,采纳 enterprise-ai-gateway 双桶设计)。
容量 = tpm_cap,回流速率 = tpm/60 每秒(连续时间回流补充);
est_tokens 为本次预扣估算(字符/3,与预扣计费同口径)。
tpm_cap <= 0 视为不限流;与 RPM 桶共用 per-key 锁,判定相互独立。
"""
if tpm_cap <= 0:
return True
now = time.monotonic()
with self._global:
lock = self._locks.setdefault(key_id, threading.Lock())
with lock:
tokens, last = self._tpm.get(key_id, (float(tpm_cap), now))
tokens = min(float(tpm_cap), tokens + (now - last) * (tpm_cap / 60.0))
if tokens < est_tokens:
self._tpm[key_id] = (tokens, now)
return False
self._tpm[key_id] = (tokens - est_tokens, now)
return True
def acquire_slot(self, key_id: int) -> bool: def acquire_slot(self, key_id: int) -> bool:
"""并发槽(非阻塞);返回 False = 超并发上限(429)。""" """并发槽(非阻塞);返回 False = 超并发上限(429)。"""
with self._global: with self._global:
+72
View File
@@ -21,6 +21,10 @@ _SQL_HOLD = """UPDATE students SET
spent_today_milli = ?, spent_date = ? spent_today_milli = ?, spent_date = ?
WHERE id = ? AND balance_milli >= ?""" WHERE id = ? AND balance_milli >= ?"""
# 预算四档阈值(T-X1,基点 bp:1% = 100bp;整数比较,杜绝浮点边界漂移)
BUDGET_OPTIMIZE_BP = 8000 # >= 80% 建议降一档
BUDGET_CHEAP_BP = 9500 # >= 95% 强制最低档
_SQL_HOLD_MARK = """INSERT INTO usage_ledger _SQL_HOLD_MARK = """INSERT INTO usage_ledger
(request_id, ts, key_id, model, bucket, charged_milli, status) (request_id, ts, key_id, model, bucket, charged_milli, status)
VALUES (?, ?, ?, ?, ?, ?, 'holding')""" VALUES (?, ?, ?, ?, ?, ?, 'holding')"""
@@ -132,6 +136,41 @@ class BillingMixin:
" WHERE request_id = ?", (status, request_id)) " WHERE request_id = ?", (status, request_id))
return True return True
# ---------- 预算四档(T-X1) ----------
def budget_mode(self, student_id: int, est_milli: int, ts: float) -> str:
"""预算档位判定:normal / optimize / cheap / block。
以「当日已 spent + 本次预估」占日上限的比例判定(基点 bp,全整数运算,
无浮点边界漂移;黄金用例锁死):
< 8000bp(80%) -> normal(现行行为)
>= 8000bp(80%) -> optimize(调用方应降一档模型)
>= 9500bp(95%) -> cheap(调用方应强制最低档)
> 10000bp(100%) -> block(try_hold 的硬拒绝语义兜底)
daily_cap_milli <= 0 视为不限额 -> 恒 normal;学生不存在 -> normal
(try_hold 才是扣费与拒绝的唯一权威,本方法只做档位建议)。
"""
today = _today(ts)
with self._lock, self._connect() as conn:
stu = conn.execute(
"SELECT daily_cap_milli, spent_today_milli, spent_date"
" FROM students WHERE id = ?", (student_id,)).fetchone()
if stu is None:
return "normal"
cap = int(stu["daily_cap_milli"] or 0)
if cap <= 0:
return "normal"
spent = int(stu["spent_today_milli"] or 0) \
if stu["spent_date"] == today else 0
projected = spent + max(0, int(est_milli))
projected_bp = projected * 10000 // cap
if projected_bp > 10000:
return "block"
if projected_bp >= BUDGET_CHEAP_BP:
return "cheap"
if projected_bp >= BUDGET_OPTIMIZE_BP:
return "optimize"
return "normal"
# ---------- 流水查询 ---------- # ---------- 流水查询 ----------
def get_usage(self, request_id: str) -> Optional[Dict[str, Any]]: def get_usage(self, request_id: str) -> Optional[Dict[str, Any]]:
with self._lock, self._connect() as conn: with self._lock, self._connect() as conn:
@@ -140,6 +179,39 @@ class BillingMixin:
(request_id,)).fetchone() (request_id,)).fetchone()
return dict(row) if row else None return dict(row) if row else None
def usage_stats(self) -> Dict[str, Any]:
"""全局流水聚合(§5.2 看板口径,2026-09 优化:SQL 下推替代全量拉取)。
返回 {requests, gateway_cached, in_hit_tok, in_miss_tok, revenue_milli,
cost_milli, by_bucket};by_bucket 键为 bucket 名。
"""
with self._lock, self._connect() as conn:
row = conn.execute(
"SELECT COUNT(*) AS requests,"
" COALESCE(SUM(gateway_cached), 0) AS gateway_cached,"
" COALESCE(SUM(in_hit_tok), 0) AS in_hit_tok,"
" COALESCE(SUM(in_miss_tok), 0) AS in_miss_tok,"
" COALESCE(SUM(charged_milli), 0) AS revenue_milli,"
" COALESCE(SUM(upstream_cost_milli), 0) AS cost_milli"
" FROM usage_ledger").fetchone()
buckets = conn.execute(
"SELECT bucket, COUNT(*) AS requests,"
" COALESCE(SUM(charged_milli), 0) AS revenue_milli,"
" COALESCE(SUM(upstream_cost_milli), 0) AS cost_milli"
" FROM usage_ledger GROUP BY bucket").fetchall()
return {
"requests": row["requests"],
"gateway_cached": row["gateway_cached"],
"in_hit_tok": row["in_hit_tok"],
"in_miss_tok": row["in_miss_tok"],
"revenue_milli": row["revenue_milli"],
"cost_milli": row["cost_milli"],
"by_bucket": {r["bucket"]: {"requests": r["requests"],
"revenue_milli": r["revenue_milli"],
"cost_milli": r["cost_milli"]}
for r in buckets},
}
def list_usage(self, student_id: Optional[int] = None, def list_usage(self, student_id: Optional[int] = None,
limit: int = 50, offset: int = 0) -> List[Dict[str, Any]]: limit: int = 50, offset: int = 0) -> List[Dict[str, Any]]:
"""流水分页(可按学生过滤,经其名下 key);按学生过滤走联表常量语句。""" """流水分页(可按学生过滤,经其名下 key);按学生过滤走联表常量语句。"""
+9
View File
@@ -52,11 +52,15 @@ class ProxyConfig:
rpm_per_key: int = 10 rpm_per_key: int = 10
day_req_cap: int = 200 day_req_cap: int = 200
concurrent_per_key: int = 2 concurrent_per_key: int = 2
tpm_per_key: int = 60000 # T-X12:每分钟 token 桶(0 = 不限)
max_body_chars: int = 60000 max_body_chars: int = 60000
semcache_enabled: bool = True semcache_enabled: bool = True
sim_threshold: float = 0.92 sim_threshold: float = 0.92
max_entries: int = 300000 max_entries: int = 300000
promote_frequency: int = 5 promote_frequency: int = 5
# T-X10(采纳 cortiq 路由签名分桶):语义缓存键的路由签名粒度
# none | capabilities(默认:vision/tools 需求不同不互串) | model(按模型隔离)
route_sig_scope: str = "capabilities"
def bucket(self, name: str) -> BucketCfg: def bucket(self, name: str) -> BucketCfg:
"""取桶配置;未知名回落 default(D-P2 缺省桶)。""" """取桶配置;未知名回落 default(D-P2 缺省桶)。"""
@@ -156,9 +160,14 @@ def build_proxy_config(settings_dict: Dict[str, Any]) -> ProxyConfig:
rpm_per_key=int(limits.get("rpm_per_key", 10) or 10), rpm_per_key=int(limits.get("rpm_per_key", 10) or 10),
day_req_cap=int(limits.get("day_req_cap", 200) or 200), day_req_cap=int(limits.get("day_req_cap", 200) or 200),
concurrent_per_key=int(limits.get("concurrent_per_key", 2) or 2), concurrent_per_key=int(limits.get("concurrent_per_key", 2) or 2),
tpm_per_key=int(limits.get("tpm_per_key", 60000) or 0),
max_body_chars=int(limits.get("max_body_chars", 60000) or 60000), max_body_chars=int(limits.get("max_body_chars", 60000) or 60000),
semcache_enabled=bool(sem.get("enabled", True)), semcache_enabled=bool(sem.get("enabled", True)),
sim_threshold=float(sem.get("sim_threshold", 0.92) or 0.92), sim_threshold=float(sem.get("sim_threshold", 0.92) or 0.92),
max_entries=int(sem.get("max_entries", 300000) or 300000), max_entries=int(sem.get("max_entries", 300000) or 300000),
promote_frequency=int(sem.get("promote_frequency", 5) or 5), promote_frequency=int(sem.get("promote_frequency", 5) or 5),
route_sig_scope=(str(sem.get("route_sig_scope"))
if str(sem.get("route_sig_scope")) in
("none", "capabilities", "model")
else "capabilities"),
) )
+70 -9
View File
@@ -8,8 +8,8 @@
工程纪律: 工程纪律:
- D-P1 毫元整数,本模块不做任何浮点运算(元换算只发生在入口参数转换)。 - D-P1 毫元整数,本模块不做任何浮点运算(元换算只发生在入口参数转换)。
- D-P10:同步实现 + 全局锁每操作连接(ReviewQueue 模式),异步调用方经 - D-P10:同步实现 + 全局锁串行访问(ReviewQueue 模式;2026-09 起复用单条
asyncio.to_thread 包装;WAL 模式(§3)。 持久连接,语义不变),异步调用方经 asyncio.to_thread 包装;WAL 模式(§3)。
- 全部数据库访问使用占位符参数化语句,语句为常量,零拼接。 - 全部数据库访问使用占位符参数化语句,语句为常量,零拼接。
""" """
from __future__ import annotations from __future__ import annotations
@@ -74,6 +74,8 @@ class Ledger(BillingMixin):
self.db_path = Path(db_path) self.db_path = Path(db_path)
self.db_path.parent.mkdir(parents=True, exist_ok=True) self.db_path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.Lock() self._lock = threading.Lock()
self._conn: Optional[sqlite3.Connection] = None
self._reset_date: Optional[str] = None # 日重置短路标记(2026-09 优化)
self._init_db() self._init_db()
# ---------- 初始化 ---------- # ---------- 初始化 ----------
@@ -83,9 +85,17 @@ class Ledger(BillingMixin):
return cls(db_path) return cls(db_path)
def _connect(self) -> sqlite3.Connection: def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self.db_path, check_same_thread=False) """共享持久连接(2026-09 优化)。
conn.row_factory = sqlite3.Row
return conn 所有操作本就串行于 self._lock 之下,复用单连接语义与逐操作新建等价,
省去每请求 3-5 次 sqlite3.connect(Windows 上开销显著);
`with conn` 只做提交/回滚不关闭,连接持续可用。
"""
if self._conn is None:
conn = sqlite3.connect(self.db_path, check_same_thread=False)
conn.row_factory = sqlite3.Row
self._conn = conn
return self._conn
def _init_db(self) -> None: def _init_db(self) -> None:
with self._lock, self._connect() as conn: with self._lock, self._connect() as conn:
@@ -168,12 +178,18 @@ class Ledger(BillingMixin):
# ---------- 日限额 ---------- # ---------- 日限额 ----------
def check_and_count(self, key_id: int, student_id: int, now: float) -> bool: def check_and_count(self, key_id: int, student_id: int, now: float) -> bool:
"""日请求限额双检:跨日重置(注入日期)+ 原子计数;超限返回 False(429)。""" """日请求限额双检:跨日重置(注入日期)+ 原子计数;超限返回 False(429)。
2026-09 优化:重置 UPDATE 每实例每日只执行一次(原每请求全表扫
students 并抢写锁);单进程部署(D-P9)下语义等价。
"""
today = _today(now) today = _today(now)
with self._lock, self._connect() as conn: with self._lock, self._connect() as conn:
conn.execute( if self._reset_date != today:
"UPDATE students SET spent_today_milli = 0, spent_date = ?" conn.execute(
" WHERE spent_date IS NOT ?", (today, today)) "UPDATE students SET spent_today_milli = 0, spent_date = ?"
" WHERE spent_date IS NOT ?", (today, today))
self._reset_date = today
row = conn.execute( row = conn.execute(
"SELECT req_today, req_date, day_cap_req FROM proxy_keys" "SELECT req_today, req_date, day_cap_req FROM proxy_keys"
" WHERE id = ?", (key_id,)).fetchone() " WHERE id = ?", (key_id,)).fetchone()
@@ -187,6 +203,51 @@ class Ledger(BillingMixin):
(used + 1, today, key_id)) (used + 1, today, key_id))
return True return True
# ---------- 语义缓存持久化(T-P6,供 semcache.SemanticCache 调用) ----------
def semcache_rows(self, limit: int = 300000) -> List[Dict[str, Any]]:
"""全量缓存行(启动重建倒排索引)。"""
with self._lock, self._connect() as conn:
rows = conn.execute(
"SELECT cache_key, q_norm, answer, model, created_ts, ttl_ts,"
" doc_version, hits FROM semcache LIMIT ?", (limit,)).fetchall()
return [dict(r) for r in rows]
def put_semcache(self, cache_key: str, bucket: str, q_norm: str, answer: str,
model: str, created_ts: int, ttl_ts: int,
doc_version: int = 1) -> None:
"""写/覆盖一条缓存(幂等)。"""
with self._lock, self._connect() as conn:
conn.execute(
"INSERT OR REPLACE INTO semcache"
"(cache_key, bucket, q_norm, answer, model, created_ts, ttl_ts,"
" doc_version, hits) VALUES (?,?,?,?,?,?,?,?,0)",
(cache_key, bucket, q_norm, answer, model, created_ts, ttl_ts,
doc_version))
def promote_semcache(self, source_key: str, alias_key: str, hits: int) -> None:
"""L2 -> L1 晋升:以别名键复制一行(原行保留审计)。"""
with self._lock, self._connect() as conn:
row = conn.execute(
"SELECT * FROM semcache WHERE cache_key = ?", (source_key,)).fetchone()
if row is None:
return
conn.execute(
"INSERT OR REPLACE INTO semcache"
"(cache_key, bucket, q_norm, answer, model, created_ts, ttl_ts,"
" doc_version, hits) VALUES (?,?,?,?,?,?,?,?,?)",
(alias_key, row["bucket"], row["q_norm"], row["answer"],
row["model"], row["created_ts"], row["ttl_ts"],
row["doc_version"], hits))
conn.execute("UPDATE semcache SET hits = ? WHERE cache_key = ?",
(hits, source_key))
def purge_semcache_expired(self, now: Optional[int] = None) -> int:
"""过期缓存清理(夜间任务顺带)。"""
now = int(now if now is not None else time.time())
with self._lock, self._connect() as conn:
cur = conn.execute("DELETE FROM semcache WHERE ttl_ts < ?", (now,))
return cur.rowcount
# ---------- 自省(测试/验收用) ---------- # ---------- 自省(测试/验收用) ----------
def table_names(self) -> List[str]: def table_names(self) -> List[str]:
"""列出已建表名(测试验收)。""" """列出已建表名(测试验收)。"""
+7 -1
View File
@@ -83,13 +83,19 @@ def is_cacheable(body: dict) -> bool:
return len(non_system) <= 1 return len(non_system) <= 1
def canonical_hash(bucket: str, doc_version: int, body: dict) -> str: def canonical_hash(bucket: str, doc_version: int, body: dict,
route_sig: str = "") -> str:
"""规则 3:缓存键 = bucket + '|' + doc_version + '|' + sha256(norm)。 """规则 3:缓存键 = bucket + '|' + doc_version + '|' + sha256(norm)。
route_sig 非空时插入为第三段(T-X10,采纳 cortiq 路由签名分桶:
不同路由意图——vision/tools 需求或模型——不互串答案);为空时保持
旧三段格式(既有调用与旧缓存条目兼容,旧条目随 TTL 自然淘汰)。
桶模板/资料前缀不参与哈希(由 bucket+doc_version 表达,资料更新 = 版本+1)。 桶模板/资料前缀不参与哈希(由 bucket+doc_version 表达,资料更新 = 版本+1)。
""" """
norm = normalize_messages(body) norm = normalize_messages(body)
digest = hashlib.sha256(norm.encode("utf-8")).hexdigest() digest = hashlib.sha256(norm.encode("utf-8")).hexdigest()
if route_sig:
return f"{bucket}|{doc_version}|{route_sig}|{digest}"
return f"{bucket}|{doc_version}|{digest}" return f"{bucket}|{doc_version}|{digest}"
+709 -43
View File
@@ -13,7 +13,8 @@ import asyncio
import json import json
import time import time
import uuid import uuid
from typing import Any, Dict, Optional from collections import deque
from typing import Any, Deque, Dict, List, Optional
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import JSONResponse, StreamingResponse from fastapi.responses import JSONResponse, StreamingResponse
@@ -30,7 +31,8 @@ from gateway.proxy.errors import (
UpstreamError, UpstreamError,
) )
from gateway.proxy.pricing import compute from gateway.proxy.pricing import compute
from gateway.proxy.upstream import UpstreamAborted, filter_usage_chunk, stream as upstream_stream from gateway.proxy.upstream import (UpstreamAborted, failover_stats as upstream_failover_stats,
filter_usage_chunk, stream as upstream_stream)
def install_error_handlers(app) -> None: def install_error_handlers(app) -> None:
@@ -45,8 +47,15 @@ def install_error_handlers(app) -> None:
headers={"WWW-Authenticate": "Bearer"} if exc.status_code == 401 else None) headers={"WWW-Authenticate": "Bearer"} if exc.status_code == 401 else None)
def build_proxy_router(cfg: ProxyConfig, pool) -> APIRouter: def build_proxy_router(cfg: ProxyConfig, pool, settings_provider=None,
"""组装代理面路由(唯一组装点)。""" review_getter=None) -> APIRouter:
"""组装代理面路由(唯一组装点)。
settings_provider:返回设置对象的回调(api.py 注入 settings_store)。
未注入时语义分析器分流不启用(None 安全,既有测试不受影响)。
review_getter:返回人工检验队列的回调(api.py 注入 get_review);
未注入时 T1 审计抽样跳过(不报错)。
"""
router = APIRouter(prefix="/proxy") router = APIRouter(prefix="/proxy")
ledger = __import__("gateway.proxy.ledger", fromlist=["Ledger"]).Ledger.init_db(cfg.db_path) ledger = __import__("gateway.proxy.ledger", fromlist=["Ledger"]).Ledger.init_db(cfg.db_path)
# ---------------- 学生面 ---------------- # ---------------- 学生面 ----------------
@@ -78,42 +87,64 @@ def build_proxy_router(cfg: ProxyConfig, pool) -> APIRouter:
_limits(), time.time()) _limits(), time.time())
if not _limits().acquire_slot(ctx["key_id"]): if not _limits().acquire_slot(ctx["key_id"]):
raise QuotaError("并发请求已达该 key 上限") raise QuotaError("并发请求已达该 key 上限")
# T-X12(采纳 enterprise-ai-gateway 双桶限流):TPM 桶——RPM 之外加
# token 维度(预扣估算 = 字符/3,与计费同口径;cfg.tpm_per_key=0 不限)
tpm_cap = int(getattr(cfg, "tpm_per_key", 0) or 0)
if tpm_cap > 0:
est_tokens = max(1, len(raw_body) // 3)
if not _limits().allow_tokens(ctx["key_id"], est_tokens, tpm_cap):
_limits().release_slot(ctx["key_id"])
raise QuotaError("该 key 的每分钟 token 额度(TPM)已用尽")
# 语义分析器 live 分流(D-G7:mode=live 才启用;任何异常不影响代理可用性) # 语义分析器 live 分流(D-G7:mode=live 才启用;任何异常不影响代理可用性)。
# T-X4 修复:settings_provider 由 api.py 注入(原实现引用未导入的
# settings_store,NameError 被 except 吞掉,分流从未实际生效)。
# 2026-09 优化:sense 运行时按配置单例化(原每请求重建 SenseStore+Grader:
# 每请求跑全量 DDL、重读工件文件,且 T-X3 决策缓存随 Grader 丢弃、命中率恒 0)。
tier_used = None tier_used = None
request_id = "px" + uuid.uuid4().hex[:12]
if settings_provider is not None:
try:
from gateway.sense.config import build_sense_config
scfg = build_sense_config(settings_provider().to_dict())
if scfg.enabled and scfg.mode == "live":
_sstore, grader = _get_sense_runtime(scfg)
qtext = "\n".join(str(m.get("content") or "")
for m in (body.get("messages") or []))
d = await grader.decide(qtext or str(body.get("model") or ""),
"proxy")
tier_used = d.tier
# 档位 -> 池条目(§2 消费方表):t1/t2/t3 映射模型替换请求模型
hint = scfg.tier_pool_hint("proxy", tier_used)
e = _entry_by_tier_hint(pool, hint)
if e is not None:
body = {**body, "model": e["model"]}
except Exception:
pass # D-G4:分级故障不影响代理可用性
try: try:
from gateway.sense.config import build_sense_config resp = await _run_chat(body, dict(request.headers), ctx, cfg, ledger, pool,
from gateway.sense.grader import Grader request_id=request_id)
from gateway.sense.observer import get_observer
from gateway.sense.store import SenseStore
scfg = build_sense_config(settings_store().to_dict())
if scfg.enabled and scfg.mode == "live":
sstore = SenseStore.init_db(scfg.db_path)
grader = Grader(scfg, sstore, get_observer(sstore))
qtext = "\n".join(str(m.get("content") or "")
for m in (body.get("messages") or []))
d = await grader.decide(qtext or str(body.get("model") or ""),
"proxy")
tier_used = d.tier
# 档位 -> 池条目(§2 消费方表):t1/t2/t3 映射模型替换请求模型
hint = scfg.tier_pool_hint("proxy", tier_used)
e = _entry_by_tier_hint(pool, hint)
if e is not None:
body = {**body, "model": e["model"]}
except Exception:
pass # D-G4:分级故障不影响代理可用性
try:
resp = await _run_chat(body, dict(request.headers), ctx, cfg, ledger, pool)
if tier_used: if tier_used:
resp.headers["x-campus-tier"] = tier_used resp.headers["x-campus-tier"] = tier_used
# T1 审计抽样(§9.4):按 review.sample_rate 入队人工核 # T-X11(采纳 cortiq shadow 旁路评审):live T1 本地应答按采样率
# 异步旁路云端评审,喂晋升表(零客户端延迟,任何异常不影响响应)
if tier_used == "T1":
_maybe_shadow_review(body, resp, pool, settings_provider, scfg)
# T1 审计抽样(§9.4):按 review.sample_rate 入队人工核;
# promo 标签供人工 verdict 回填晋升表(T-X5)。
# 2026-09 修复:原块引用未定义的 load_config/get_review/request_id,
# 三个 NameError 被 except 吞掉,抽样从未入队;review 队列现由
# api.py 经 review_getter 注入(避免 proxy -> api 反向依赖)。
try: try:
import random as _random import random as _random
from router_system.config import load_config
from gateway.sense.promotion import promotion_label
rate = float(load_config().get("review", {}).get("sample_rate", 0.1)) rate = float(load_config().get("review", {}).get("sample_rate", 0.1))
if _random.random() < rate: if review_getter is not None and _random.random() < rate:
get_review().enqueue( review_getter().enqueue(
request_id + "-sense", str(body.get("model") or "proxy"), request_id + "-sense", str(body.get("model") or "proxy"),
"(sense T1 审计抽样)", tags=["sense_t1"], "(sense T1 审计抽样)",
tags=["sense_t1", "promo:" + promotion_label("proxy", "")],
reason="sense_audit") reason="sense_audit")
except Exception: except Exception:
pass pass
@@ -123,7 +154,7 @@ def build_proxy_router(cfg: ProxyConfig, pool) -> APIRouter:
def _entry_by_tier_hint(pool, hint: str) -> Optional[Dict[str, Any]]: def _entry_by_tier_hint(pool, hint: str) -> Optional[Dict[str, Any]]:
"""按档位名选池条目(tier_hint 优先,缺省按 tier 字段映射)。""" """按档位名选池条目(tier_hint 优先,缺省按 tier 字段映射)。"""
entries = pool.list().get("entries", []) entries = _pool_entries(pool)
for e in entries: for e in entries:
if not e.get("enabled") or e.get("backend") == "mock": if not e.get("enabled") or e.get("backend") == "mock":
continue continue
@@ -173,10 +204,87 @@ def build_proxy_router(cfg: ProxyConfig, pool) -> APIRouter:
@router.post("/admin/keys/{key_id}/revoke", tags=["proxy-admin"]) @router.post("/admin/keys/{key_id}/revoke", tags=["proxy-admin"])
async def admin_revoke_key(key_id: int, request: Request): async def admin_revoke_key(key_id: int, request: Request):
_guard_admin(request) _guard_admin(request)
from gateway.proxy.auth import _AUTH_SINGLETON
ok = ledger.revoke_key(key_id) ok = ledger.revoke_key(key_id)
return {"ok": ok} return {"ok": ok}
@router.get("/admin/stats", tags=["proxy-admin"])
async def admin_stats(request: Request, since: int = 0):
"""命中率-毛利看板(§5.2 口径):h_g/h_p/revenue/cost/margin/by_bucket。
2026-09 优化:SQL 聚合下推(原 list_usage(limit=500000) 全量拉回
Python 扫 4 遍,随流水线性劣化);响应字段与四舍五入口径不变。
"""
_guard_admin(request)
from gateway.proxy.ledgerutil import _today
import time as _t
now = _t.time()
today = _today(now)
s = ledger.usage_stats()
n = s["requests"]
cached = s["gateway_cached"]
h_g = cached / n if n else 0.0
in_hit = s["in_hit_tok"]
in_miss = s["in_miss_tok"]
h_p = in_hit / (in_hit + in_miss) if (in_hit + in_miss) else 0.0
# T-X12:进程内语义缓存计数透出(entries/hits/misses/hit_rate)
sem_stats = None
if cfg.semcache_enabled:
try:
sem_stats = _get_semcache(cfg, ledger).stats()
except Exception: # noqa: BLE001
sem_stats = None
return {"requests": n, "h_g": round(h_g, 4), "h_p": round(h_p, 4),
"revenue_milli": s["revenue_milli"], "cost_milli": s["cost_milli"],
"margin_milli": s["revenue_milli"] - s["cost_milli"],
"by_bucket": s["by_bucket"],
"semcache": sem_stats,
"route_scored": list(_route_score_events)[-10:][::-1],
"today": today,
"upstream_failover": upstream_failover_stats()}
@router.get("/admin/sense-decisions", tags=["proxy-admin"])
async def admin_sense_decisions(request: Request, limit: int = 50):
"""路由决策留痕查询(T-X4):reasons/candidate_scores/rejected 三元结构。
管理面鉴权同 /admin/stats;sense 未启用时返回空集(不报错,便于前端
统一渲染)。
"""
_guard_admin(request)
try:
from gateway.sense.config import build_sense_config
from gateway.sense.store import SenseStore
if settings_provider is None:
return {"enabled": False, "decisions": []}
scfg = build_sense_config(settings_provider().to_dict())
if not scfg.enabled:
return {"enabled": False, "decisions": []}
sstore = SenseStore.init_db(scfg.db_path)
return {"enabled": True,
"decisions": sstore.list_decisions(limit=limit)}
except Exception:
return {"enabled": False, "decisions": []}
@router.get("/admin/ledger", tags=["proxy-admin"])
async def admin_ledger(request: Request, student_id: int = 0,
limit: int = 50, offset: int = 0):
"""流水分页(§5.2)。"""
_guard_admin(request)
return ledger.list_usage(student_id=student_id or None,
limit=max(1, min(limit, 200)),
offset=max(0, offset))
@router.get("/admin/students", tags=["proxy-admin"])
async def admin_list_students(request: Request):
"""学生列表(key 管理卡片)。
注:学生表直读查询因安全扫描误报暂缓入库(ledger.list_students 待补),
当前列表由签发/充值时的写入响应累积(前端本地态);端点先返回 501。
"""
_guard_admin(request)
return JSONResponse({"error": {"message": "学生列表查询待补(安全扫描误报阻塞)",
"type": "not_implemented"}},
status_code=501)
return router return router
@@ -185,6 +293,15 @@ def _limits():
return _AUTH_SINGLETON_LIMITS return _AUTH_SINGLETON_LIMITS
def _pool_entries(pool) -> List[Dict[str, Any]]:
"""池条目快照(转发链内部专用):优先未打码通道 usable_entries,
回退打码 list()(兼容测试注入的简化 pool)。HTTP 响应一律用 list()。"""
fn = getattr(pool, "usable_entries", None)
if fn is not None:
return fn()
return pool.list().get("entries", [])
def _estimate_hold_milli(body: dict, cfg: ProxyConfig, model: str) -> int: def _estimate_hold_milli(body: dict, cfg: ProxyConfig, model: str) -> int:
"""预扣估算(宁可高估,D-P11):in 按字符/3,out 按 min(max_tokens,4096)。 """预扣估算(宁可高估,D-P11):in 按字符/3,out 按 min(max_tokens,4096)。
@@ -217,20 +334,524 @@ def _resolve_entry(pool, model: str, cfg: ProxyConfig) -> Optional[Dict[str, Any
return None return None
def _budget_headers(budget_mode: str) -> Dict[str, str]:
"""预算档位响应头(normal 不发,避免噪音)。"""
return {"X-Budget-Mode": budget_mode} if budget_mode and budget_mode != "normal" else {}
def _request_needs(body: dict) -> Dict[str, Any]:
"""从请求体推断能力需求(T-X6):多模态图片 -> vision;带 tools -> tools;
上下文需求 = 提示字符/3 + min(max_tokens, 4096)(与预扣估算同口径)。"""
vision = False
for m in body.get("messages") or []:
content = m.get("content")
if isinstance(content, list):
for part in content:
if isinstance(part, dict) and ("image_url" in part
or part.get("type") == "image_url"):
vision = True
prompt_chars = sum(len(str(m.get("content") or ""))
if not isinstance(m.get("content"), list)
else sum(len(str(p.get("text") or "")) for p in m["content"]
if isinstance(p, dict))
for m in (body.get("messages") or []))
est_out = min(int(body.get("max_tokens") or 1024), 4096)
return {"vision": vision,
"tools": bool(body.get("tools")),
"min_context_tokens": prompt_chars // 3 + est_out}
def _entry_meets(entry: Dict[str, Any], needs: Dict[str, Any]) -> bool:
"""条目能力位是否满足请求需求(capabilities 缺省全兼容)。"""
cap = entry.get("capabilities") or {}
if needs["vision"] and not cap.get("vision", True):
return False
if needs["tools"] and not cap.get("tools", True):
return False
ctx = int(cap.get("context_window") or 0)
if needs["min_context_tokens"] > 0 and 0 < ctx < needs["min_context_tokens"]:
return False
return True
def _fallback_chain(pool, entry: Dict[str, Any], max_total: int = 3) -> List[Dict[str, Any]]:
"""上游有序降级链(T-X2,采纳 cortiq tier 链思路)。
主条目之外,取池内其他启用真实后端条目,按 (档位升序, 单价和升序) 排列
——便宜的先顶上;总链长 <= max_total。仅作首 token 前 failover 候选
(D-P4),不做负载均衡(单写者模型)。
"""
from gateway.model_pool import TIERS
tier_rank = {t: i for i, t in enumerate(TIERS)}
rest = [e for e in _pool_entries(pool)
if (e.get("enabled") and e.get("id") != entry.get("id")
and e.get("backend") not in ("mock",) and e.get("base_url")
and e.get("model") != entry.get("model"))]
rest.sort(key=lambda e: (tier_rank.get(e.get("tier"), 99),
float(e.get("price_in") or 0) + float(e.get("price_out") or 0)))
return rest[:max(0, int(max_total) - 1)]
def _downgrade_entry(pool, entry: Dict[str, Any], mode: str) -> Optional[Dict[str, Any]]:
"""预算降档(T-X1):在池内找恰好低一档/最低档的启用条目。
档位序 TIERS = (local, budget, premium):
- optimize:降一档(premium->budget,budget->local);
- cheap:直落最低档 local;
- 已在 local / 找不到该档启用条目 -> None(保持原条目,档位语义照常上报)。
"""
from gateway.model_pool import TIERS
tier = str(entry.get("tier") or "")
if tier not in TIERS:
return None
rank = TIERS.index(tier)
if mode == "cheap":
target = 0
elif mode == "optimize":
target = rank - 1
else:
return None
if target < 0:
return None
want = TIERS[target]
for e in _pool_entries(pool):
if (e.get("enabled") and e.get("id") != entry.get("id")
and e.get("tier") == want
and e.get("backend") not in ("mock",) and e.get("base_url")):
return e
return None
_semcache_instances: Dict[str, Any] = {}
def _get_semcache(cfg: ProxyConfig, ledger):
"""按 db_path 的进程内缓存单例(D-P9 单进程前提;不同库隔离)。"""
inst = _semcache_instances.get(cfg.db_path)
if inst is None:
from gateway.proxy.semcache import SemanticCache
inst = SemanticCache(
ledger, max_entries=cfg.max_entries,
sim_threshold=cfg.sim_threshold,
promote_frequency=cfg.promote_frequency)
_semcache_instances[cfg.db_path] = inst
return inst
def reset_semcache_instances() -> None:
"""测试用:清空缓存单例。"""
global _semcache_instances
_semcache_instances = {}
_sense_runtimes: Dict[tuple, Any] = {}
def _get_sense_runtime(scfg):
"""sense 运行时按配置单例化(2026-09 优化,激活 T-X3 决策缓存)。
键取 SenseConfig 的完整 repr:任何设置变更都会派生新键并重建运行时,
旧实例留在注册表中随设置维度有界增长(单进程部署,D-P9)。
返回 (sstore, grader):Grader 持有的线性头/阈值/60s 决策缓存跨请求复用,
免去每请求的 SenseStore DDL 与工件文件重读。
"""
key = ("sense", repr(scfg))
rt = _sense_runtimes.get(key)
if rt is None:
from gateway.sense.grader import Grader
from gateway.sense.observer import get_observer
from gateway.sense.promotion import PromotionTable
from gateway.sense.store import SenseStore
sstore = SenseStore.init_db(scfg.db_path)
grader = Grader(scfg, sstore, get_observer(sstore),
promotion=PromotionTable(sstore))
rt = (sstore, grader)
_sense_runtimes[key] = rt
return rt
def reset_sense_runtimes() -> None:
"""测试用:清空 sense 运行时单例。"""
global _sense_runtimes
_sense_runtimes = {}
# ---------------------------------------------------------------------------
# T-X11(采纳 cortiq shadow 旁路评审设计):live T1 本地应答的异步云端评审。
# 按采样率旁路:premium 条目给本地答案打 PASS/FAIL,写入 T-X5 晋升表——
# 打通晋升表的质量信号自动来源。零客户端延迟(asyncio.create_task 旁路)。
# ---------------------------------------------------------------------------
_shadow_review_stats: Dict[str, int] = {"sampled": 0, "pass": 0, "fail": 0,
"error": 0, "skipped_busy": 0}
_shadow_sem: Optional[asyncio.Semaphore] = None
_SHADOW_CONCURRENCY = 2 # 旁路评审并发上限(保护上游)
_SHADOW_MAX_ANSWER_CHARS = 4000 # 送审答案截断
def reset_shadow_review_stats() -> None:
"""测试用:清零旁路评审计数并复位并发闸门。"""
global _shadow_review_stats, _shadow_sem
_shadow_review_stats = {"sampled": 0, "pass": 0, "fail": 0,
"error": 0, "skipped_busy": 0}
_shadow_sem = None
def _shadow_review_settings(settings_dict: dict) -> Dict[str, Any]:
"""解析 sense.shadow_review 设置(默认关闭;fail-safe 兜底值)。"""
raw = (settings_dict.get("sense") or {}).get("shadow_review") or {}
if not isinstance(raw, dict):
raw = {}
try:
rate = max(0.0, min(1.0, float(raw.get("sample_rate", 0.1) or 0.1)))
except (TypeError, ValueError):
rate = 0.1
try:
timeout_s = max(1.0, float(raw.get("timeout_s", 20) or 20))
except (TypeError, ValueError):
timeout_s = 20.0
return {"enabled": bool(raw.get("enabled", False)),
"sample_rate": rate, "timeout_s": timeout_s,
"tier": str(raw.get("tier", "premium") or "premium")}
def _pick_review_entry(pool, tier_hint: str) -> Optional[Dict[str, Any]]:
"""选评审条目:优先指定档位,回退任意启用真实后端条目。"""
entries = [e for e in _pool_entries(pool)
if e.get("enabled") and e.get("backend") not in ("mock",)
and e.get("base_url")]
for e in entries:
if (e.get("tier") or "") == tier_hint:
return e
return entries[0] if entries else None
async def _grade_answer(entry, question: str, answer: str, model: str,
timeout_s: float) -> str:
"""用指定池条目评审本地答案,返回 PASS / FAIL / ERROR(上游故障不抛出)。"""
judge_body = {
"model": model,
"messages": [
{"role": "system", "content": "你是严格的答案质量评审员。只输出 PASS 或 FAIL。"},
{"role": "user",
"content": (f"问题:{question}\n\n候选答案:{answer[:_SHADOW_MAX_ANSWER_CHARS]}\n\n"
"该答案是否正确且切题?只输出一个词:PASS 或 FAIL。")},
],
"max_tokens": 8,
}
parts: List[str] = []
sink: Dict[str, Any] = {}
async def _collect():
async for raw in upstream_stream(judge_body, entry, sink, [entry]):
line = raw.decode("utf-8", errors="replace").strip()
for sub in line.split("\n\n"):
if not sub.startswith("data:"):
continue
payload = sub[5:].strip()
if not payload or payload == "[DONE]":
continue
try:
obj = json.loads(payload)
except json.JSONDecodeError:
continue
delta = (obj.get("choices") or [{}])[0].get("delta") or {}
if delta.get("content"):
parts.append(str(delta["content"]))
try:
await asyncio.wait_for(_collect(), timeout=timeout_s)
except Exception: # noqa: BLE001 超时/上游异常:评审失败不算 FAIL
return "ERROR"
text = "".join(parts).strip().upper()
if "PASS" in text:
return "PASS"
if "FAIL" in text:
return "FAIL"
return "ERROR"
async def _shadow_review_task(question: str, answer: str, pool,
settings_dict: dict, promotion,
tier_used: str = "T1") -> None:
"""旁路评审任务体:调用方 create_task 丢弃即可,任何异常不影响主链路。"""
cfgs = _shadow_review_settings(settings_dict)
entry = _pick_review_entry(pool, cfgs["tier"])
if entry is None:
return
global _shadow_sem
if _shadow_sem is None:
_shadow_sem = asyncio.Semaphore(_SHADOW_CONCURRENCY)
try:
async with _shadow_sem:
verdict = await _grade_answer(entry, question, answer,
str(entry.get("model") or ""),
cfgs["timeout_s"])
except Exception: # noqa: BLE001
verdict = "ERROR"
_shadow_review_stats["sampled"] += 1
if verdict in ("PASS", "FAIL"):
_shadow_review_stats["pass" if verdict == "PASS" else "fail"] += 1
try:
from gateway.sense.promotion import promotion_label
promotion.observe(promotion_label("proxy", ""),
ok=(verdict == "PASS"), tier=tier_used)
except Exception: # noqa: BLE001
pass
else:
_shadow_review_stats["error"] += 1
def _maybe_shadow_review(body: dict, resp, pool, settings_provider, scfg) -> None:
"""主链路挂钩:live T1 且命中采样率时旁路评审(仅非流式响应可取答案文本)。"""
try:
import random as _random
cfgs = _shadow_review_settings(settings_provider().to_dict())
if not cfgs["enabled"] or _random.random() >= cfgs["sample_rate"]:
return
if isinstance(resp, StreamingResponse):
return # 流式响应此刻无完整答案文本,跳过(后续迭代可经 sink 采集)
payload = json.loads(bytes(resp.body))
answer = str((((payload.get("choices") or [{}])[0])
.get("message") or {}).get("content") or "")
if not answer:
return
question = "\n".join(str(m.get("content") or "")
for m in (body.get("messages") or []))
_, grader = _get_sense_runtime(scfg)
promotion = grader._promotion # 同包内复用晋升表(T-X5)
if promotion is None:
return
asyncio.get_running_loop().create_task(
_shadow_review_task(question, answer, pool,
settings_provider().to_dict(), promotion))
except Exception: # noqa: BLE001 旁路失败不影响主响应(D-G4 纪律)
pass
def _route_sig(needs: Dict[str, Any], model: str, cfg: ProxyConfig) -> str:
"""路由签名(T-X10,采纳 cortiq 语义缓存路由签名分桶)。
签名进入缓存键,使不同路由意图的请求不互串答案:
- capabilities(默认):vision/tools 需求不同 -> 不同签名(多模态/工具请求
不再命中纯文本缓存答案——正确性优先);
- model:按模型名隔离(更保守,命中率换绝对隔离);
- none:空签名(旧行为,最高命中率)。
"""
scope = getattr(cfg, "route_sig_scope", "capabilities")
if scope == "model":
return f"m:{model}"
if scope == "capabilities":
return f"v{int(bool(needs['vision']))}t{int(bool(needs['tools']))}"
return ""
# ---------------------------------------------------------------------------
# T-X13(采纳 ai-model-router 五维评分 + 硬过滤拒绝理由设计):能力位硬过滤
# (T-X6,语义不变)之后对存活候选做确定性加权评分——改变的是链内顺序而非
# 过滤语义;被淘汰候选带人话拒绝理由进评分留痕(管理面可观测)。
# ---------------------------------------------------------------------------
_ROUTE_SCORE_WEIGHTS = {
"cost_efficiency": 0.30, "capability": 0.25, "speed": 0.20,
"reliability": 0.15, "quality": 0.10,
}
_route_score_events: Deque[Dict[str, Any]] = deque(maxlen=100)
def reset_route_score_events() -> None:
"""测试用:清空评分留痕。"""
_route_score_events.clear()
def _entry_reject_reason(entry: Dict[str, Any], needs: Dict[str, Any]) -> Optional[str]:
"""能力位不满足时的人话拒绝理由(满足则 None)。"""
cap = entry.get("capabilities") or {}
if needs["vision"] and not cap.get("vision", True):
return "模型不支持视觉输入(vision)"
if needs["tools"] and not cap.get("tools", True):
return "模型不支持工具调用(tools)"
ctx = int(cap.get("context_window") or 0)
if needs["min_context_tokens"] > 0 and 0 < ctx < needs["min_context_tokens"]:
return f"上下文窗口 {ctx} 小于需求 {needs['min_context_tokens']}"
return None
def _score_entry(entry: Dict[str, Any], needs: Dict[str, Any]) -> Dict[str, Any]:
"""单候选五维评分(各维归一 0-1;确定性)。
cost_efficiency = 1/(1+均价*2);speed = 1 - 档位秩比(local 最快);
quality = 档位秩比(premium 质量最高的启发式);capability = 满足需求时
按上下文余量给分、不满足为 0;reliability 预留(单写者模型暂无按模型
失败率统计,恒 1.0,权重占位)。
"""
from gateway.model_pool import TIERS
price = max(0.0, (float(entry.get("price_in") or 0)
+ float(entry.get("price_out") or 0)) / 2.0)
try:
rank = TIERS.index(str(entry.get("tier") or ""))
except ValueError:
rank = max(0, len(TIERS) - 1)
span = max(1, len(TIERS) - 1)
cap = entry.get("capabilities") or {}
ctx = int(cap.get("context_window") or 0)
meets = _entry_meets(entry, needs)
headroom = 1.0 if ctx <= 0 else min(1.0, ctx / max(1, needs["min_context_tokens"]))
dims = {
"cost_efficiency": 1.0 / (1.0 + price * 2.0),
"capability": headroom if meets else 0.0,
"speed": 1.0 - rank / span,
"reliability": 1.0,
"quality": rank / span,
}
total = sum(dims[k] * _ROUTE_SCORE_WEIGHTS[k] for k in dims)
return {"model": str(entry.get("model") or ""), "tier": entry.get("tier"),
"total": round(total, 4),
"dims": {k: round(v, 4) for k, v in dims.items()}}
def _rank_candidates(candidates: List[Dict[str, Any]], needs: Dict[str, Any]
) -> List[Dict[str, Any]]:
"""候选按 total 降序(同分按模型名字典序,确定性)。"""
return sorted((_score_entry(e, needs) for e in candidates),
key=lambda s: (-s["total"], s["model"]))
def _record_route_score_event(request_id: str, ranked: List[Dict[str, Any]],
rejected: List[Dict[str, Any]]) -> None:
"""评分留痕:入选排序 + 被淘汰候选的拒绝理由(管理面可观测)。"""
_route_score_events.append({
"ts": int(time.time()), "request_id": request_id,
"ranked": ranked[:3],
"rejected": rejected,
})
def _cached_charge(body: dict, cfg: ProxyConfig, model: str) -> Dict[str, int]:
"""缓存命中计费:成本 0,按未命中口径对入/出估 token 收售价(§7)。"""
from gateway.proxy.pricing import compute
usage = {"in_miss": len(json.dumps(body.get("messages") or "",
ensure_ascii=False)) // 3,
"in_hit": 0, "out": 0}
return compute(usage, model, time.time(), cfg)
async def _run_chat(body: dict, headers: Dict[str, str], ctx: Dict[str, Any], async def _run_chat(body: dict, headers: Dict[str, str], ctx: Dict[str, Any],
cfg: ProxyConfig, ledger, pool): cfg: ProxyConfig, ledger, pool,
request_id: Optional[str] = None):
model = str(body.get("model") or "") model = str(body.get("model") or "")
entry = _resolve_entry(pool, model, cfg) entry = _resolve_entry(pool, model, cfg)
if entry is None: if entry is None:
return JSONResponse({"error": {"message": f"模型不可用: {model}", return JSONResponse({"error": {"message": f"模型不可用: {model}",
"type": "invalid_request_error"}}, "type": "invalid_request_error"}},
status_code=400) status_code=400)
request_id = "px" + uuid.uuid4().hex[:12] if request_id is None:
request_id = "px" + uuid.uuid4().hex[:12]
ts = time.time() ts = time.time()
client_wants_usage = bool(body.get("stream_options", {}).get("include_usage")) \ client_wants_usage = bool(body.get("stream_options", {}).get("include_usage")) \
if isinstance(body.get("stream_options"), dict) else False if isinstance(body.get("stream_options"), dict) else False
is_stream = bool(body.get("stream")) is_stream = bool(body.get("stream"))
# ---- 能力位硬过滤(T-X6):请求需要 vision/tools/上下文而条目不满足时重定向 ----
needs = _request_needs(body)
if not _entry_meets(entry, needs):
from gateway.model_pool import filter_by_capabilities
all_entries = [e for e in _pool_entries(pool)
if e.get("enabled") and e.get("backend") not in ("mock",)
and e.get("base_url")]
cap_entries = filter_by_capabilities(
list(all_entries),
need_vision=needs["vision"], need_tools=needs["tools"],
min_context_tokens=needs["min_context_tokens"])
if cap_entries:
# T-X13:存活候选五维评分,取最高者为重定向目标(不再取首元素)
ranked = _rank_candidates(cap_entries, needs)
best_model = ranked[0]["model"]
entry = next(e for e in cap_entries
if str(e.get("model") or "") == best_model)
model = str(entry.get("model") or model)
body = {**body, "model": model}
_record_route_score_event(
request_id, ranked,
[{"model": str(e.get("model") or ""),
"reason": _entry_reject_reason(e, needs) or "评分落选"}
for e in all_entries if not _entry_meets(e, needs)])
# ---- 预算四档(T-X1):接近日上限渐进降档;>100% 仍由 try_hold 硬拒 ----
budget_mode = "normal"
try:
est_probe = _estimate_hold_milli(body, cfg, model)
budget_mode = await asyncio.to_thread(
ledger.budget_mode, ctx["student_id"], est_probe, ts)
except Exception:
budget_mode = "normal"
if budget_mode in ("optimize", "cheap"):
down = _downgrade_entry(pool, entry, budget_mode)
if down is not None:
entry = down
model = str(entry.get("model") or model)
body = {**body, "model": model}
# ---- 缓存分支(T-P6,§7 时序):仅缓存准入(stop+单轮)查询 ----
# T-X10:缓存键带路由签名(scope 由 cfg.route_sig_scope 决定),且
# lookup/put 共用同一 norm_hash(原先 put 侧重复计算一次)
cacheable = False
cache = None
norm_hash = ""
try:
if cfg.semcache_enabled:
from gateway.proxy.normalizer import canonical_hash, is_cacheable
from gateway.proxy.semcache import SemanticCache
bucket_cfg = cfg.bucket(str(headers.get("x-campus-bucket") or "default"))
cacheable = is_cacheable(body)
if cacheable:
sig = _route_sig(needs, model, cfg)
norm_hash = canonical_hash(bucket_cfg.name, bucket_cfg.doc_version,
body, sig)
cache = _get_semcache(cfg, ledger)
norm_text = json.dumps(body.get("messages") or [], ensure_ascii=False,
sort_keys=True)
hit = cache.lookup(norm_hash, norm_text, bucket_cfg.doc_version,
route_sig=sig)
if hit is not None:
est = _estimate_hold_milli(body, cfg, model)
if await asyncio.to_thread(
ledger.try_hold, request_id, ctx["key_id"],
ctx["student_id"], model, bucket_cfg.name, est, ts):
br = _cached_charge(body, cfg, model)
await asyncio.to_thread(
ledger.settle, request_id, br["charged_milli"],
gateway_cached=1, upstream_cost_milli=0,
ttfb_ms=0, status="cached")
if is_stream:
from gateway.proxy.semcache import synth_sse_chunks
chunks = synth_sse_chunks(hit["answer"], model=model,
request_id=request_id)
async def replay():
for c in chunks:
yield c
return StreamingResponse(
replay(), media_type="text/event-stream",
headers={"Cache-Control": "no-cache",
"X-Cache": "HIT",
"X-Request-Id": request_id,
**_budget_headers(budget_mode)})
return JSONResponse({
"id": f"chatcmpl-{request_id}", "object": "chat.completion",
"created": int(time.time()), "model": model,
"choices": [{"index": 0,
"message": {"role": "assistant",
"content": hit["answer"]},
"finish_reason": "stop"}],
"usage": {"prompt_tokens": 0, "completion_tokens": 0,
"total_tokens": 0},
}, headers={"X-Cache": "HIT", "X-Request-Id": request_id,
**_budget_headers(budget_mode)})
raise BalanceError("余额或当日额度不足")
except BalanceError:
raise
except Exception:
cache = None # 缓存层故障不影响主流程(降级直连上游)
est = _estimate_hold_milli(body, cfg, model) est = _estimate_hold_milli(body, cfg, model)
if not await asyncio.to_thread( if not await asyncio.to_thread(
ledger.try_hold, request_id, ctx["key_id"], ctx["student_id"], ledger.try_hold, request_id, ctx["key_id"], ctx["student_id"],
@@ -239,12 +860,25 @@ async def _run_chat(body: dict, headers: Dict[str, str], ctx: Dict[str, Any],
t0 = time.perf_counter() t0 = time.perf_counter()
sink: Dict[str, Any] = {} sink: Dict[str, Any] = {}
chain = _fallback_chain(pool, entry)
# T-X13:降级链按五维评分降序重排(稳定:同分保持原相对序;评分异常回退原链)
try:
totals = {s["model"]: s["total"] for s in _rank_candidates(chain, needs)}
chain = sorted(chain, key=lambda e: (-totals.get(str(e.get("model") or ""), 0.0),
str(e.get("model") or "")))
except Exception: # noqa: BLE001
pass
try: try:
if is_stream: if is_stream:
return await _stream_response(body, entry, sink, headers, client_wants_usage, return await _stream_response(body, entry, sink, headers, client_wants_usage,
request_id, ctx, cfg, ledger, model, est, t0) request_id, ctx, cfg, ledger, model, est, t0,
cache=cache, cacheable=cacheable,
budget_mode=budget_mode, chain=chain)
return await _json_response(body, entry, sink, request_id, ctx, cfg, return await _json_response(body, entry, sink, request_id, ctx, cfg,
ledger, model, est, t0) ledger, model, est, t0,
cache=cache, cacheable=cacheable,
headers=headers, budget_mode=budget_mode,
chain=chain, norm_hash=norm_hash)
except UpstreamAborted as e: except UpstreamAborted as e:
# 流中失败:按已收 usage 结算(无 usage 按字符估算),不缓存(D-P4) # 流中失败:按已收 usage 结算(无 usage 按字符估算),不缓存(D-P4)
usage = sink.get("usage") or _estimate_usage_from_sink(sink) usage = sink.get("usage") or _estimate_usage_from_sink(sink)
@@ -272,7 +906,9 @@ def _estimate_usage_from_sink(sink: Dict[str, Any]) -> Dict[str, int]:
async def _stream_response(body, entry, sink, headers, client_wants_usage, async def _stream_response(body, entry, sink, headers, client_wants_usage,
request_id, ctx, cfg, ledger, model, est, t0): request_id, ctx, cfg, ledger, model, est, t0,
cache=None, cacheable=False, budget_mode: str = "normal",
chain: Optional[List[Dict[str, Any]]] = None):
usage = {"in_miss": 0, "in_hit": 0, "out": 0} usage = {"in_miss": 0, "in_hit": 0, "out": 0}
async def gen(): async def gen():
@@ -280,7 +916,8 @@ async def _stream_response(body, entry, sink, headers, client_wants_usage,
chunk_id = f"chatcmpl-{request_id}" chunk_id = f"chatcmpl-{request_id}"
created = int(time.time()) created = int(time.time())
try: try:
async for raw_bytes in upstream_stream(body, entry, sink, [entry]): async for raw_bytes in upstream_stream(body, entry, sink,
chain if chain is not None else [entry]):
line = raw_bytes.decode("utf-8").strip() line = raw_bytes.decode("utf-8").strip()
if not line: if not line:
continue continue
@@ -319,13 +956,18 @@ async def _stream_response(body, entry, sink, headers, client_wants_usage,
return StreamingResponse(gen(), media_type="text/event-stream", return StreamingResponse(gen(), media_type="text/event-stream",
headers={"Cache-Control": "no-cache", headers={"Cache-Control": "no-cache",
"X-Request-Id": request_id}) "X-Request-Id": request_id,
**_budget_headers(budget_mode)})
async def _json_response(body, entry, sink, request_id, ctx, cfg, ledger, async def _json_response(body, entry, sink, request_id, ctx, cfg, ledger,
model, est, t0): model, est, t0, cache=None, cacheable=False,
headers=None, budget_mode: str = "normal",
chain: Optional[List[Dict[str, Any]]] = None,
norm_hash: str = ""):
parts = [] parts = []
async for raw_bytes in upstream_stream(body, entry, sink, [entry]): async for raw_bytes in upstream_stream(body, entry, sink,
chain if chain is not None else [entry]):
line = raw_bytes.decode("utf-8").strip() line = raw_bytes.decode("utf-8").strip()
if not line: if not line:
continue continue
@@ -353,6 +995,29 @@ async def _json_response(body, entry, sink, request_id, ctx, cfg, ledger,
in_miss_tok=usage.get("in_miss", 0), in_hit_tok=usage.get("in_hit", 0), in_miss_tok=usage.get("in_miss", 0), in_hit_tok=usage.get("in_hit", 0),
out_tok=usage.get("out", 0), upstream_cost_milli=br["upstream_cost_milli"], out_tok=usage.get("out", 0), upstream_cost_milli=br["upstream_cost_milli"],
ttfb_ms=sink.get("ttfb_ms"), total_ms=total_ms, status="ok") ttfb_ms=sink.get("ttfb_ms"), total_ms=total_ms, status="ok")
# 缓存准入(D-P5):stop 且单轮且未命中来的 -> 写缓存
# T-X10:键复用 lookup 时的 norm_hash(含路由签名),不再重复计算
if cache is not None and cacheable and norm_hash \
and sink.get("finish_reason", "stop") == "stop":
try:
from gateway.proxy.semcache import _sig_from_key
bucket_cfg = cfg.bucket(str(headers.get("x-campus-bucket")
or "default")) if headers else cfg.bucket("default")
norm_text = json.dumps(body.get("messages") or [], ensure_ascii=False,
sort_keys=True)
cache.put(norm_hash, norm_text, "".join(parts), model,
doc_version=bucket_cfg.doc_version,
ttl_hours=bucket_cfg.ttl_hours,
route_sig=_sig_from_key(norm_hash))
except Exception:
pass
fb = sink.get("upstream_fallback") or {}
fb_headers: Dict[str, str] = {}
if fb.get("used"):
fb_headers = {"X-Upstream-Fallback": "1",
"X-Upstream-Original": str(fb.get("original") or ""),
"X-Upstream-Used": str(fb.get("used_model") or ""),
"X-Upstream-Reason": str(fb.get("reason") or "")[:200]}
return JSONResponse({ return JSONResponse({
"id": f"chatcmpl-{request_id}", "id": f"chatcmpl-{request_id}",
"object": "chat.completion", "object": "chat.completion",
@@ -366,4 +1031,5 @@ async def _json_response(body, entry, sink, request_id, ctx, cfg, ledger,
"completion_tokens": usage.get("out", 0), "completion_tokens": usage.get("out", 0),
"total_tokens": usage.get("in_miss", 0) + usage.get("in_hit", 0) "total_tokens": usage.get("in_miss", 0) + usage.get("in_hit", 0)
+ usage.get("out", 0)}, + usage.get("out", 0)},
}, headers={"X-Request-Id": request_id}) }, headers={"X-Request-Id": request_id,
**_budget_headers(budget_mode), **fb_headers})
+306 -13
View File
@@ -1,24 +1,317 @@
"""语义缓存(T-P6 落地;本文件先立签名)。 """两级语义缓存(T-P6,M2):L1 精确 + L2 n-gram 倒排 + singleflight + SSE 回放。
规格(§6):L1 精确 + L2 字符 2/3-gram 倒排(启动自 sqlite 重建), 规格(§6,测试最重):
加权 Jaccard(3-gram 权 2)+ 共享 gram>=3 门限 + 阈值 0.92, - L1:cache_key(bucket|doc_version|sha256(norm))-> 条目,LRU(max_entries,默认 30 万)
TTL + LRU(max_entries),L2 命中 promote_frequency 次晋升 L1。 - L2:字符 2-gram + 3-gram **集合**,内存倒排索引 gram -> [cache_key];
启动时由 semcache 表 q_norm 重建;加权 Jaccard(3-gram 权 2、2-gram 权 1);
候选门限:共享 gram >= 3 才计分;>= sim_threshold(0.92) 命中;
**L2 命中累计 promote_frequency(5) 次晋升 L1**
- TTL:ttl_ts 过期不可见;命中即续期(滑动过期)
- 持久化:semcache 表(put 同步写,索引内存维护;调用方 to_thread,D-P10)
性能设计(2026-09 优化):
- 加权 Jaccard 以 w(A∪B) = w(A) + w(B) − w(A∩B) 免构建并集集合;
条目权重在写入时预计算(CacheEntry.w),查询权重每次查找算一次
- 候选先做规模上界预筛:w_inter ≤ min(wA,wB) 且 w_union ≥ max(wA,wB),
min/max < 阈值者不可能命中,免相交计算(不影响可命中集合)
""" """
from __future__ import annotations from __future__ import annotations
from typing import Any, Dict, Optional import asyncio
import hashlib
import json
import re
import time
from collections import OrderedDict
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
_WHITESPACE = re.compile(r"\s+")
def grams(text: str) -> set:
"""字符 2-gram + 3-gram 集合(中文天然适配,无需分词)。"""
t = _WHITESPACE.sub("", (text or "").lower())
out: set = set()
for n in (2, 3):
for i in range(len(t) - n + 1):
out.add(t[i:i + n])
return out or ({t} if t else set())
def _weight(g: set) -> int:
"""gram 集合的加权规模:3-gram 权 2、2-gram 权 1(恒为非负整数)。"""
return sum(2 if len(x) == 3 else 1 for x in g)
def weighted_jaccard(ga: set, gb: set) -> float:
"""加权 Jaccard:交集中每个 3-gram 权 2、2-gram 权 1,除以并集加权。
等价公式:w_inter / (w(ga) + w(gb) − w_inter),权重和为整数,
浮点结果与逐项遍历并集的旧实现完全一致。
"""
if not ga or not gb:
return 0.0
inter = ga & gb
if not inter:
return 0.0
w_inter = _weight(inter)
w_union = _weight(ga) + _weight(gb) - w_inter
return w_inter / w_union if w_union else 0.0
class CacheEntry:
__slots__ = ("answer", "model", "q_norm", "g", "w", "created_ts", "ttl_ts",
"doc_version", "hits", "sig")
def __init__(self, answer: str, model: str, q_norm: str,
created_ts: float, ttl_ts: float, doc_version: int,
sig: str = ""):
self.answer = answer
self.model = model
self.q_norm = q_norm
self.g = grams(q_norm)
self.w = _weight(self.g) # 预计算加权规模,查询期免重算
self.created_ts = created_ts
self.ttl_ts = ttl_ts
self.doc_version = doc_version
self.hits = 0
self.sig = sig # 路由签名(T-X10:L2 扫描按签名分区)
def _sig_from_key(cache_key: str) -> str:
"""从缓存键解析路由签名(T-X10)。
带签名键 = bucket|doc_version|sig|sha256(4 段);旧三段键无签名返回 ""。
签名段约束不含 '|'(capabilities/model 两种 scope 均满足),故解析无歧义。
"""
parts = cache_key.split("|")
return parts[2] if len(parts) >= 4 else ""
class SemanticCache: class SemanticCache:
"""两级语义缓存(T-P6 实现)。""" """L1 精确 + L2 倒排(内存),sqlite 持久化(store 的 semcache 表)。"""
def lookup(self, bucket: str, doc_version: int, norm_hash: str, def __init__(self, store, max_entries: int = 300000,
norm_text: str, now: float) -> Optional[Dict[str, Any]]: sim_threshold: float = 0.92, promote_frequency: int = 5,
raise NotImplementedError("T-P6") now: Optional[float] = None):
self.store = store
self.max_entries = max(1, int(max_entries))
self.sim_threshold = float(sim_threshold)
self.promote_frequency = max(1, int(promote_frequency))
self._l1: "OrderedDict[str, CacheEntry]" = OrderedDict()
self._inverted: Dict[str, set] = {}
self._clock = now # 假时钟注入(None = time.time)
self.hits_exact = 0
self.hits_semantic = 0
self.misses = 0
self._rebuild()
def put(self, bucket: str, doc_version: int, norm_hash: str, # ---------- 时钟 ----------
norm_text: str, answer: str, model: str, now: float) -> None: def _now(self) -> float:
raise NotImplementedError("T-P6") return time.time() if self._clock is None else self._clock
# ---------- 启动重建 ----------
def _rebuild(self) -> None:
try:
rows = self.store.semcache_rows(limit=self.max_entries)
except Exception: # noqa: BLE001
return
for r in rows:
entry = CacheEntry(r["answer"], r["model"], r["q_norm"],
r["created_ts"], r["ttl_ts"], r["doc_version"],
sig=_sig_from_key(r["cache_key"]))
entry.hits = r["hits"]
self._index(r["cache_key"], entry, promote=False)
def _index(self, cache_key: str, entry: CacheEntry, promote: bool = True) -> None:
self._l1[cache_key] = entry
self._l1.move_to_end(cache_key)
for g in entry.g:
self._inverted.setdefault(g, set()).add(cache_key)
if promote:
self._evict()
def _evict(self) -> None:
"""LRU 驱逐(超 max_entries 淘汰最久未用,含倒排回收)。"""
while len(self._l1) > self.max_entries:
key, entry = self._l1.popitem(last=False)
for g in entry.g:
bucket = self._inverted.get(g)
if bucket is not None:
bucket.discard(key)
if not bucket:
self._inverted.pop(g, None)
# ---------- 查询 ----------
def lookup(self, cache_key: str, norm_text: str,
doc_version: int = 1,
route_sig: Optional[str] = None) -> Optional[Dict[str, Any]]:
"""L1 精确 -> L2 语义(§7 签名)。返回 {answer, model, level} 或 None。
route_sig 非 None 时按签名分区(T-X10):L1 命中要求条目签名一致,
L2 扫描跳过签名不一致的候选(不同路由意图不互串答案);
None = 不校验(旧调用/旧测试完全兼容)。
"""
now = self._now()
# L1
entry = self._l1.get(cache_key)
if entry is not None and (route_sig is None or entry.sig == route_sig):
if entry.ttl_ts < now:
self._invalidate_key(cache_key)
else:
self.hits_exact += 1
self._l1.move_to_end(cache_key)
return {"answer": entry.answer, "model": entry.model,
"level": "exact"}
# L2
g = grams(norm_text)
if len(g) < 3:
self.misses += 1
return None
w_q = _weight(g)
candidates: Dict[str, int] = {}
for gram in g:
for key in self._inverted.get(gram, ()):
candidates[key] = candidates.get(key, 0) + 1
best_key = None
best_score = 0.0
for key, shared in candidates.items():
if shared < 3:
continue # 候选门限:共享 gram >= 3
cand = self._l1.get(key)
if cand is None or cand.ttl_ts < now or cand.doc_version != doc_version:
continue
if route_sig is not None and cand.sig != route_sig:
continue # 签名分区:跨路由意图不参与计分
# 规模上界预筛:w_inter <= lo 且 w_union >= hi,故 score <= lo/hi;
# 严格小于阈值者不可能命中,跳过(不构建相交集合)。
# 注意用严格不等式:lo/hi == 阈值的边界候选仍会进入精确计分,
# 保证命中集合与"全量计分"完全一致。
lo, hi = (w_q, cand.w) if w_q <= cand.w else (cand.w, w_q)
if lo / hi < self.sim_threshold:
continue
w_inter = _weight(g & cand.g)
score = w_inter / (w_q + cand.w - w_inter)
if score > best_score:
best_score = score
best_key = key
if best_key is None or best_score < self.sim_threshold:
self.misses += 1
return None
cand = self._l1[best_key]
cand.hits += 1
self.hits_semantic += 1
promoted = False
if cand.hits >= self.promote_frequency:
# L2 -> L1:生成精确键(由 q_norm 重建 cache_key 由调用方语义保证一致——
# 这里以 sha256(q_norm) 前缀别名入 L1,桶/版本/签名由 cand 自带)
h = hashlib.sha256(cand.q_norm.encode()).hexdigest()[:16]
if cand.sig:
alias = f"{best_key.split('|')[0]}|{cand.doc_version}|{cand.sig}|promoted:{h}"
else:
alias = f"{best_key.split('|')[0]}|{cand.doc_version}|promoted:{h}"
self._index(alias, cand, promote=True)
try:
self.store.promote_semcache(best_key, alias, cand.hits)
except Exception: # noqa: BLE001
pass
promoted = True
_ = promoted
return {"answer": cand.answer, "model": cand.model, "level": "semantic"}
def _invalidate_key(self, cache_key: str) -> None:
entry = self._l1.pop(cache_key, None)
if entry:
for g in entry.g:
bucket = self._inverted.get(g)
if bucket is not None:
bucket.discard(cache_key)
if not bucket:
self._inverted.pop(g, None)
# ---------- 写入 ----------
def put(self, cache_key: str, q_norm: str, answer: str, model: str,
doc_version: int = 1, ttl_hours: int = 72,
route_sig: str = "") -> None:
"""写 L1 + 倒排 + sqlite 持久化(§7 签名)。route_sig 随条目留存供 L2 分区。"""
now = self._now()
entry = CacheEntry(answer, model, q_norm, now, now + ttl_hours * 3600,
doc_version, sig=route_sig)
self._index(cache_key, entry)
try:
self.store.put_semcache(cache_key, "default", q_norm, answer, model,
int(now), int(now + ttl_hours * 3600),
doc_version)
except Exception: # noqa: BLE001
pass # 持久化失败不影响内存缓存(可重建)
def stats(self) -> Dict[str, Any]: def stats(self) -> Dict[str, Any]:
raise NotImplementedError("T-P6") total = self.hits_exact + self.hits_semantic + self.misses
return {"entries": len(self._l1), "hits_exact": self.hits_exact,
"hits_semantic": self.hits_semantic, "misses": self.misses,
"hit_rate": round((self.hits_exact + self.hits_semantic) / total, 4)
if total else 0.0}
class SingleFlight:
"""同请求合并(§7):dict[norm_hash -> Future],上限 256,60s 超时降级。"""
MAX = 256
TIMEOUT_S = 60.0
def __init__(self):
self._inflight: Dict[str, asyncio.Future] = {}
def try_claim(self, norm_hash: str):
"""返回 (future_or_None, slot)。future 非 None = 等待方;slot = 登记句柄。"""
fut = self._inflight.get(norm_hash)
if fut is not None:
return fut, None
if len(self._inflight) >= self.MAX:
return None, None # 超限旁路(不合并)
fut = asyncio.get_running_loop().create_future()
self._inflight[norm_hash] = fut
return None, (norm_hash, fut)
def release(self, slot, result: Any = None, error: Any = None) -> None:
if slot is None:
return
norm_hash, fut = slot
self._inflight.pop(norm_hash, None)
if not fut.done():
if error is not None:
fut.set_exception(error)
else:
fut.set_result(result)
async def wait(self, fut, timeout_s: float = TIMEOUT_S):
"""等待方:超时 -> 降级直连(返回 None)。"""
try:
return await asyncio.wait_for(asyncio.shield(fut), timeout=timeout_s)
except (asyncio.TimeoutError, Exception):
return None
# ---------------- SSE 合成回放(§6) ----------------
def synth_sse_chunks(answer: str, chunk_size: int = 20,
model: str = "cached", request_id: str = "") -> List[bytes]:
"""缓存命中且 stream=true:合成合法 SSE(分块 delta + finish + [DONE])。"""
cid = f"chatcmpl-cached-{request_id or '0'}"
created = int(time.time())
out: List[bytes] = []
for i in range(0, max(1, len(answer)), chunk_size):
piece = answer[i:i + chunk_size]
out.append(("data: " + json.dumps({
"id": cid, "object": "chat.completion.chunk", "created": created,
"model": model,
"choices": [{"index": 0, "delta": {"content": piece}, "finish_reason": None}],
}, ensure_ascii=False) + "\n\n").encode("utf-8"))
out.append(("data: " + json.dumps({
"id": cid, "object": "chat.completion.chunk", "created": created,
"model": model,
"choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
}, ensure_ascii=False) + "\n\n").encode("utf-8"))
out.append(b"data: [DONE]\n\n")
return out
+47 -2
View File
@@ -11,7 +11,9 @@
from __future__ import annotations from __future__ import annotations
import json import json
from typing import Any, AsyncIterator, Dict, List, Optional import time
from collections import deque
from typing import Any, AsyncIterator, Deque, Dict, List, Optional
import httpx import httpx
@@ -42,6 +44,36 @@ class UpstreamAborted(Exception):
"""首 token 已下发后上游失败(D-P4:不可 failover,按已收 usage 计费)。""" """首 token 已下发后上游失败(D-P4:不可 failover,按已收 usage 计费)。"""
# ---------------- failover 统计(T-X2:如实上报,进程内,D-P9 单进程) ----------------
_FAILOVER_EVENTS_MAX = 100
_failover_events: Deque[Dict[str, Any]] = deque(maxlen=_FAILOVER_EVENTS_MAX)
_failover_total = 0
def _record_failover(original: str, failed_model: str, reason: str) -> None:
"""记录一次「首 token 前失败 -> 切换候选」事件。"""
global _failover_total
_failover_total += 1
_failover_events.append({
"ts": int(time.time()), "original": original,
"failed_model": failed_model, "reason": reason,
})
def failover_stats(recent: int = 20) -> Dict[str, Any]:
"""failover 统计(/proxy/admin/stats 聚合暴露)。"""
events = list(_failover_events)
return {"total": _failover_total,
"recent": events[-max(0, int(recent)):][::-1]}
def reset_failover_stats() -> None:
"""测试用:清空 failover 统计。"""
global _failover_total
_failover_events.clear()
_failover_total = 0
def normalize_usage(provider: str, usage_dict: Dict[str, Any]) -> Dict[str, int]: def normalize_usage(provider: str, usage_dict: Dict[str, Any]) -> Dict[str, int]:
"""三家 usage 字段 -> 统一 {in_miss, in_hit, out}。 """三家 usage 字段 -> 统一 {in_miss, in_hit, out}。
@@ -89,17 +121,22 @@ def filter_usage_chunk(raw_line: str, client_wants_usage: bool) -> Optional[str]
async def stream(body: dict, entry: Dict[str, Any], usage_sink: Dict[str, Any], async def stream(body: dict, entry: Dict[str, Any], usage_sink: Dict[str, Any],
fallback_entries: Optional[List[Dict[str, Any]]] = None fallback_entries: Optional[List[Dict[str, Any]]] = None
) -> AsyncIterator[bytes]: ) -> AsyncIterator[bytes]:
"""流式派发(§6 签名扩展 failover 链)。 """流式派发(§6 签名扩展 failover 链;T-X2 扩展降级上报)。
- 始终注入 include_usage;usage chunk 交给 usage_sink(透传过滤由调用方用 - 始终注入 include_usage;usage chunk 交给 usage_sink(透传过滤由调用方用
filter_usage_chunk 决定)。 filter_usage_chunk 决定)。
- D-P4:首 token 前(未 yield 任何字节)失败 -> 切换 fallback_entries; - D-P4:首 token 前(未 yield 任何字节)失败 -> 切换 fallback_entries;
已 yield 后失败 -> 抛 UpstreamAborted。 已 yield 后失败 -> 抛 UpstreamAborted。
- usage_sink["usage"] 收敛为归一化 dict;sink["ttfb_ms"] 记录首字节耗时。 - usage_sink["usage"] 收敛为归一化 dict;sink["ttfb_ms"] 记录首字节耗时。
- T-X2 如实上报:failover 发生时记 sink["upstream_fallback"]
{used, original, used_model, reason},并计入模块级 failover 统计
(failover_stats / reset_failover_stats),/proxy/admin/stats 聚合暴露。
""" """
import time as _time import time as _time
candidates: List[Dict[str, Any]] = [entry] + list(fallback_entries or []) candidates: List[Dict[str, Any]] = [entry] + list(fallback_entries or [])
last_err: Optional[Exception] = None last_err: Optional[Exception] = None
head_model = str(candidates[0].get("model") or "")
first_failure: Optional[Dict[str, Any]] = None
for cand in candidates: for cand in candidates:
provider = str(cand.get("provider") or "openai") provider = str(cand.get("provider") or "openai")
@@ -130,10 +167,18 @@ async def stream(body: dict, entry: Dict[str, Any], usage_sink: Dict[str, Any],
first = True first = True
usage_sink["ttfb_ms"] = int((_time.perf_counter() - t0) * 1000) usage_sink["ttfb_ms"] = int((_time.perf_counter() - t0) * 1000)
yield (raw + "\n\n").encode("utf-8") yield (raw + "\n\n").encode("utf-8")
if first_failure is not None:
first_failure["used_model"] = str(cand.get("model") or "")
usage_sink["upstream_fallback"] = dict(first_failure)
return # 正常结束 return # 正常结束
except Exception as exc: # noqa: BLE001 except Exception as exc: # noqa: BLE001
if first: if first:
raise UpstreamAborted(str(exc)) from exc raise UpstreamAborted(str(exc)) from exc
last_err = exc last_err = exc
if first_failure is None:
first_failure = {"used": True, "original": head_model,
"used_model": "", "reason": f"{type(exc).__name__}: {exc}"}
_record_failover(head_model, str(cand.get("model") or ""),
f"{type(exc).__name__}: {exc}")
continue # 首 token 前失败 -> failover continue # 首 token 前失败 -> failover
raise UpstreamError(f"上游均不可用: {type(last_err).__name__}: {last_err}") raise UpstreamError(f"上游均不可用: {type(last_err).__name__}: {last_err}")
+74
View File
@@ -0,0 +1,74 @@
"""路由决策缓存(T-X3,采纳 cortiq router_client 的决策哈希缓存思路)。
- key = sha256(scope + 0x00 + 归一文本);LRU + TTL(默认 60s / 4096 条,
对齐 auth._AuthCache 的进程内模式,D-P9 单进程前提)。
- 只缓存**纯决策负载**(如 probs/head_version/vec);调用方的观察与审计
落盘职责不因缓存命中而跳过。
- 时钟可注入(now(monotonic)),供测试确定性。
"""
from __future__ import annotations
import hashlib
import threading
import time
from collections import OrderedDict
from typing import Any, Callable, Optional
_DEFAULT_TTL_S = 60.0
_DEFAULT_MAX = 4096
class DecisionCache:
"""进程内决策缓存:LRU + TTL,键为 (scope, 归一文本) 的 sha256。"""
def __init__(self, maxsize: int = _DEFAULT_MAX, ttl_s: float = _DEFAULT_TTL_S,
now: Optional[Callable[[], float]] = None):
self._max = max(1, int(maxsize))
self._ttl = float(ttl_s)
self._now = now or time.monotonic
self._data: "OrderedDict[str, tuple[float, Any]]" = OrderedDict()
self._lock = threading.Lock()
self.hits = 0
self.misses = 0
@staticmethod
def key(scope: str, text: str) -> str:
"""缓存键:scope 隔离消费方/模式,文本归一由调用方完成。"""
return hashlib.sha256(f"{scope}\x00{text}".encode("utf-8")).hexdigest()
def get(self, scope: str, text: str) -> Optional[Any]:
now = self._now()
k = self.key(scope, text)
with self._lock:
item = self._data.get(k)
if item is None:
self.misses += 1
return None
ts, payload = item
if now - ts > self._ttl:
self._data.pop(k, None)
self.misses += 1
return None
self._data.move_to_end(k)
self.hits += 1
return payload
def put(self, scope: str, text: str, payload: Any) -> None:
k = self.key(scope, text)
with self._lock:
self._data[k] = (self._now(), payload)
self._data.move_to_end(k)
while len(self._data) > self._max:
self._data.popitem(last=False)
def clear(self) -> None:
"""清空条目(阈值/工件版本切换时由 Grader.invalidate 调用)。"""
with self._lock:
self._data.clear()
def stats(self) -> dict:
with self._lock:
return {"entries": len(self._data), "hits": self.hits,
"misses": self.misses,
"hit_rate": round(self.hits / (self.hits + self.misses), 4)
if (self.hits + self.misses) else 0.0}
+124 -19
View File
@@ -17,9 +17,12 @@ from typing import Any, Dict, List, Optional
from gateway.sense.calibrate import load_active from gateway.sense.calibrate import load_active
from gateway.sense.classifier import LinearHead from gateway.sense.classifier import LinearHead
from gateway.sense.config import SenseConfig from gateway.sense.config import SenseConfig
from gateway.sense.decision_cache import DecisionCache
from gateway.sense.embedder import embed from gateway.sense.embedder import embed
from gateway.sense.errors import EmbedderDown from gateway.sense.errors import EmbedderDown
from gateway.sense.features import gate from gateway.sense.features import gate
from gateway.sense.promotion import promotion_label
from gateway.sense.store import json_dumps
@dataclass @dataclass
@@ -43,16 +46,69 @@ def _rule_tier(feats) -> str:
return "T2" return "T2"
def _decision_reasons(fallback: bool, probs: Dict[str, float],
th: Dict[str, Any], feats) -> List[str]:
"""决策理由(T-X4):按 §8 判定顺序产出,可解释路由的最小完备集。"""
if fallback:
r = ["fallback:rule_gate"]
if feats.repo_signals:
r.append("repo_signals")
if feats.t1_hard_ok:
r.append("t1_hard_ok")
return r
p1 = probs.get("t1", 0.0) >= float(th.get("t1", 0.9))
p3 = probs.get("t3", 0.0) >= float(th.get("t3", 0.9))
t1_ok = p1 and feats.t1_hard_ok
t3_ok = p3 or feats.repo_signals
r: List[str] = []
if t3_ok and not t1_ok:
r.append("t3_ok")
elif t1_ok:
r.append("t1_ok")
else:
r.append("default_t2")
if p1:
r.append("p1>=threshold")
if not feats.t1_hard_ok:
r.append("gate:t1_hard_ok=0")
if p3:
r.append("p3>=threshold")
if feats.repo_signals:
r.append("repo_signals")
return r
def _rejected_tiers(fallback: bool, probs: Dict[str, float],
th: Dict[str, Any], feats) -> List[Dict[str, str]]:
"""落选档位及原因(T2 为缺省档不记录;fallback 时两条均给 fallback 原因)。"""
if fallback:
return [{"tier": "T1", "reason": "fallback"},
{"tier": "T3", "reason": "fallback"}]
p1 = probs.get("t1", 0.0) >= float(th.get("t1", 0.9))
p3 = probs.get("t3", 0.0) >= float(th.get("t3", 0.9))
t1_ok = p1 and feats.t1_hard_ok
t3_ok = p3 or feats.repo_signals
out: List[Dict[str, str]] = []
if not t3_ok:
out.append({"tier": "T3", "reason": "p3<threshold 且无 repo_signals"})
if not t1_ok:
out.append({"tier": "T1",
"reason": "p1<threshold" if not p1 else "t1_hard_ok=0"})
return out
class Grader: class Grader:
"""决策器(持有 active 工件缓存;工件/阈值切换后调 invalidate)。""" """决策器(持有 active 工件缓存;工件/阈值切换后调 invalidate)。"""
def __init__(self, cfg: SenseConfig, store, observer=None): def __init__(self, cfg: SenseConfig, store, observer=None, promotion=None):
self.cfg = cfg self.cfg = cfg
self.store = store self.store = store
self.observer = observer self.observer = observer
self._promotion = promotion # T-X5:晋升表(可 None)
self._head: Optional[LinearHead] = None self._head: Optional[LinearHead] = None
self._head_loaded = False self._head_loaded = False
self._thresholds: Optional[Dict[str, Any]] = None self._thresholds: Optional[Dict[str, Any]] = None
self._dcache = DecisionCache() # T-X3:live 模式决策缓存
def _load_head(self): def _load_head(self):
if not self._head_loaded: if not self._head_loaded:
@@ -66,10 +122,11 @@ class Grader:
return self._head return self._head
def invalidate(self) -> None: def invalidate(self) -> None:
"""工件 promote 后调用(重载 active 工件与阈值)。""" """工件 promote 后调用(重载 active 工件与阈值;同步清空决策缓存)。"""
self._head = None self._head = None
self._head_loaded = False self._head_loaded = False
self._thresholds = None self._thresholds = None
self._dcache.clear()
def _thresholds_cached(self) -> Dict[str, Any]: def _thresholds_cached(self) -> Dict[str, Any]:
if self._thresholds is None: if self._thresholds is None:
@@ -79,32 +136,47 @@ class Grader:
async def decide(self, query_or_messages, consumer: str, async def decide(self, query_or_messages, consumer: str,
domain: str = "", request_id: str = "", domain: str = "", request_id: str = "",
executed_tier: str = "") -> TierDecision: executed_tier: str = "") -> TierDecision:
"""分级决策(§8 时序;全模式写观察)。""" """分级决策(§8 时序;全模式写观察)。
T-X3:live 模式下对 (consumer, 文本) 的纯决策负载(probs/head_version/vec)
做 60s LRU 缓存,命中时跳过 embed + 线性头;观察/审计照常落盘。
collect/shadow 模式不走缓存(校准数据必须全量产出)。
"""
ts = time.time() ts = time.time()
rid = request_id or ("rt" + uuid.uuid4().hex[:10]) rid = request_id or ("rt" + uuid.uuid4().hex[:10])
text = (query_or_messages if isinstance(query_or_messages, str)
else "\n".join(str(m.get("content") or "")
for m in query_or_messages))
feats = gate(query_or_messages, consumer, self.cfg) feats = gate(query_or_messages, consumer, self.cfg)
probs: Dict[str, float] = {} probs: Dict[str, float] = {}
head_version = "" head_version = ""
fallback = False fallback = False
vec: Optional[List[int]] = None vec: Optional[List[int]] = None
try: cached = self._dcache.get(consumer, text) \
vec = await embed(feats and (query_or_messages if self.cfg.mode == "live" else None
if isinstance(query_or_messages, str) if cached is not None:
else "\n".join( probs = dict(cached["probs"])
str(m.get("content") or "") head_version = str(cached["head_version"])
for m in query_or_messages)), vec = cached["vec"]
self.cfg.embedder) else:
except EmbedderDown: try:
fallback = True vec = await embed(text, self.cfg.embedder)
except EmbedderDown:
fallback = True
head = None if fallback else self._load_head() head = None if fallback else self._load_head()
if head is None: if head is None:
fallback = True fallback = True
if not fallback and vec is not None: if not fallback and vec is not None:
probs = head.predict([float(v) for v in vec]) probs = head.predict([float(v) for v in vec])
head_version = head.version head_version = head.version
if self.cfg.mode == "live":
self._dcache.put(consumer, text,
{"probs": dict(probs),
"head_version": head_version,
"vec": vec})
th = self._thresholds_cached() th = self._thresholds_cached()
th_version = str(th.get("version") or "") th_version = str(th.get("version") or "")
@@ -124,6 +196,18 @@ class Grader:
else: else:
tier = "T2" tier = "T2"
# ---- 晋升表应用(T-X5:conformal 之外的统计闸门;仅 live)----
# 仅对 T2 决策且无任何 T3 信号时,允许已晋升标签升级 T1;
# T1/T3 决策与 fallback 路径不受晋升表影响。
promotion_applied = False
if (self._promotion is not None and self.cfg.mode == "live"
and not fallback and tier == "T2"
and not feats.repo_signals
and probs.get("t3", 0.0) < float(th.get("t3", 0.9))):
if self._promotion.is_promoted(promotion_label(consumer, domain)):
tier = "T1"
promotion_applied = True
# ---- mode 裁剪(D-G7)---- # ---- mode 裁剪(D-G7)----
if self.cfg.mode == "live": if self.cfg.mode == "live":
executed = tier # live:决策即执行 executed = tier # live:决策即执行
@@ -138,11 +222,32 @@ class Grader:
"intent_blocked": feats.intent_blocked, "intent_blocked": feats.intent_blocked,
"repo_signals": feats.repo_signals, "repo_signals": feats.repo_signals,
"over_length": feats.over_length, "over_length": feats.over_length,
"t1_hard_ok": feats.t1_hard_ok}, "t1_hard_ok": feats.t1_hard_ok,
"promotion_applied": promotion_applied},
thresholds_version=th_version, head_version=head_version, thresholds_version=th_version, head_version=head_version,
mode=self.cfg.mode, fallback=fallback, mode=self.cfg.mode, fallback=fallback,
executed_tier=executed) executed_tier=executed)
# ---- 决策留痕(T-X4:reasons / candidate_scores / rejected 三元结构)----
# 2026-09 优化:同步 sqlite 写移入线程池(原直接跑在事件循环线程上,
# 高并发下阻塞整个网关;观察写本就有批量队列,决策留痕此处补齐同等保护)
try:
import asyncio
row = {
"ts": ts, "request_id": rid, "consumer": consumer,
"decided_tier": tier, "executed_tier": executed,
"mode": self.cfg.mode, "fallback": fallback,
"policy_version": f"head:{head_version}|th:{th_version}",
"q_hash": __import__("hashlib").sha256(
text.encode("utf-8")).hexdigest()[:32],
"reasons": json_dumps(_decision_reasons(fallback, probs, th, feats)),
"candidate_scores": json_dumps(probs),
"rejected": json_dumps(_rejected_tiers(fallback, probs, th, feats)),
}
await asyncio.to_thread(self.store.insert_decision, row)
except Exception: # noqa: BLE001
pass # 留痕失败不影响决策(观察与代理主链路优先)
# ---- 观察必写(全模式)---- # ---- 观察必写(全模式)----
if self.observer is not None: if self.observer is not None:
self.observer.log(__import__("gateway.sense.observer", self.observer.log(__import__("gateway.sense.observer",
+81
View File
@@ -0,0 +1,81 @@
"""云端评判式晋升表(T-X5,采纳 cortiq promotion.rs,与 conformal 双闸门)。
定位:conformal 阈值给出「分对率 >= 1-α」的统计保证(单条查询粒度),
本表在其之上给出「某类任务(label)可由本地档接管」的**节奏自动化**:
- 观察:人工核验 verdict(approve=ok)等质量信号按 label 累计通过率;
- 晋升:n_total >= n_min 且 通过率 >= promote_lb 且 soak 浸泡期满足
-> state=promoted(此后 grader 对该 label 的 T2 决策可升级 T1);
- 退化:promoted 期间通过率 < demote_lb -> demoted(自动回退分级路径);
- 一票否决:tier=T3(高复杂度/仓库级)观察不进通过率统计,且若已晋升
立即降级——与 cortiq "HIGH tier escalation is served by the cloud" 同构。
灰度纪律(沿用 D-G7):collect 数据不足(n_min 未满)不得晋升;
全部状态迁移可由 T-X4 的决策留痕审计。
"""
from __future__ import annotations
import time
from typing import Any, Dict, Optional
def promotion_label(consumer: str, domain: str = "") -> str:
"""晋升粒度:消费方×领域(domain 缺省归 general;调用方可传更细粒度)。"""
return f"{consumer or 'default'}:{domain or 'general'}"
class PromotionTable:
"""标签晋升状态机(candidate / promoted / demoted;持久化于 sense_promotion)。"""
def __init__(self, store, now=None, n_min: int = 20, promote_lb: float = 0.95,
soak_days: float = 3.0, demote_lb: float = 0.85,
high_tier_veto: bool = True):
self.store = store
self._now = now or (lambda: time.time())
self.n_min = max(1, int(n_min))
self.promote_lb = float(promote_lb)
self.demote_lb = float(demote_lb)
self.soak_s = float(soak_days) * 86400.0
self.high_tier_veto = bool(high_tier_veto)
def observe(self, label: str, ok: bool, tier: str = "",
ts: Optional[float] = None) -> Dict[str, Any]:
"""记录一次质量信号并推进状态机;返回迁移后的行。"""
ts = float(ts) if ts is not None else float(self._now())
row = self.store.get_promotion(label) or {
"label": label, "n_total": 0, "n_ok": 0, "state": "candidate",
"first_ts": int(ts), "promoted_ts": 0, "last_ts": 0}
# 一票否决:T3 观察不进通过率统计;已晋升者立即降级
if self.high_tier_veto and str(tier).upper() == "T3":
row["last_ts"] = int(ts)
if row["state"] == "promoted":
row["state"] = "demoted"
row["promoted_ts"] = 0
self.store.upsert_promotion(row)
return row
row["n_total"] = int(row["n_total"]) + 1
row["n_ok"] = int(row["n_ok"]) + (1 if ok else 0)
row["last_ts"] = int(ts)
rate = row["n_ok"] / row["n_total"] if row["n_total"] else 0.0
soaked = (ts - int(row["first_ts"])) >= self.soak_s
if row["state"] == "promoted":
if row["n_total"] >= self.n_min and rate < self.demote_lb:
row["state"] = "demoted" # 退化自动降级
row["promoted_ts"] = 0
else:
# candidate / demoted 共用同一晋升门(降级后可凭数据恢复)
if row["n_total"] >= self.n_min and rate >= self.promote_lb and soaked:
row["state"] = "promoted"
row["promoted_ts"] = int(ts)
self.store.upsert_promotion(row)
return row
def is_promoted(self, label: str) -> bool:
row = self.store.get_promotion(label)
return bool(row) and row["state"] == "promoted"
def snapshot(self, label: str) -> Optional[Dict[str, Any]]:
return self.store.get_promotion(label)
+79
View File
@@ -7,6 +7,7 @@ WAL;全部访问经全局锁 + 每操作新连接(ReviewQueue 模式),
from __future__ import annotations from __future__ import annotations
import sqlite3 import sqlite3
import json
import threading import threading
import time import time
from pathlib import Path from pathlib import Path
@@ -29,6 +30,25 @@ CREATE TABLE IF NOT EXISTS sense_artifacts(
version TEXT PRIMARY KEY, kind TEXT NOT NULL, version TEXT PRIMARY KEY, kind TEXT NOT NULL,
path TEXT NOT NULL, metrics TEXT NOT NULL, created_ts INTEGER NOT NULL, path TEXT NOT NULL, metrics TEXT NOT NULL, created_ts INTEGER NOT NULL,
active INTEGER DEFAULT 0); active INTEGER DEFAULT 0);
CREATE TABLE IF NOT EXISTS sense_decisions(
id INTEGER PRIMARY KEY, ts INTEGER NOT NULL,
request_id TEXT NOT NULL, consumer TEXT NOT NULL,
decided_tier TEXT NOT NULL, executed_tier TEXT NOT NULL DEFAULT '',
mode TEXT NOT NULL DEFAULT 'collect', fallback INTEGER NOT NULL DEFAULT 0,
policy_version TEXT DEFAULT '', q_hash TEXT DEFAULT '',
reasons TEXT NOT NULL DEFAULT '[]',
candidate_scores TEXT NOT NULL DEFAULT '{}',
rejected TEXT NOT NULL DEFAULT '[]');
CREATE INDEX IF NOT EXISTS idx_dec_ts ON sense_decisions(ts);
CREATE TABLE IF NOT EXISTS sense_promotion(
label TEXT PRIMARY KEY, n_total INTEGER NOT NULL DEFAULT 0,
n_ok INTEGER NOT NULL DEFAULT 0,
state TEXT NOT NULL DEFAULT 'candidate',
first_ts INTEGER NOT NULL DEFAULT 0,
promoted_ts INTEGER NOT NULL DEFAULT 0,
last_ts INTEGER NOT NULL DEFAULT 0);
""" """
@@ -131,6 +151,65 @@ class SenseStore:
"DELETE FROM tier_observations WHERE ts < ?", (int(ts),)) "DELETE FROM tier_observations WHERE ts < ?", (int(ts),))
return cur.rowcount return cur.rowcount
# ---------- 决策留痕(T-X4:可解释路由三元结构) ----------
def insert_decision(self, rec: Dict[str, Any]) -> None:
"""写入一条路由决策记录(reasons/candidate_scores/rejected 为 JSON 串)。"""
with self._lock, self._connect() as conn:
conn.execute(
"""INSERT INTO sense_decisions
(ts, request_id, consumer, decided_tier, executed_tier, mode,
fallback, policy_version, q_hash, reasons, candidate_scores,
rejected)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?)""",
(int(rec.get("ts", time.time())), rec["request_id"],
rec["consumer"], rec["decided_tier"],
rec.get("executed_tier", ""), rec.get("mode", "collect"),
1 if rec.get("fallback") else 0,
rec.get("policy_version", ""), rec.get("q_hash", ""),
rec.get("reasons", "[]"), rec.get("candidate_scores", "{}"),
rec.get("rejected", "[]")))
def list_decisions(self, limit: int = 50) -> List[Dict[str, Any]]:
"""最近 N 条决策(新在前;JSON 字段反序列化为对象)。"""
with self._lock, self._connect() as conn:
rows = conn.execute(
"SELECT * FROM sense_decisions ORDER BY id DESC LIMIT ?",
(max(1, int(limit)),)).fetchall()
out: List[Dict[str, Any]] = []
for r in rows:
d = dict(r)
for f in ("reasons", "candidate_scores", "rejected"):
try:
d[f] = json.loads(d.get(f) or ("{}" if f == "candidate_scores" else "[]"))
except json.JSONDecodeError:
pass
out.append(d)
return out
# ---------- 晋升表(T-X5:本地档自动接管的双闸门之一) ----------
def upsert_promotion(self, row: Dict[str, Any]) -> None:
with self._lock, self._connect() as conn:
conn.execute(
"""INSERT OR REPLACE INTO sense_promotion
(label, n_total, n_ok, state, first_ts, promoted_ts, last_ts)
VALUES (?,?,?,?,?,?,?)""",
(row["label"], int(row.get("n_total", 0)),
int(row.get("n_ok", 0)), str(row.get("state", "candidate")),
int(row.get("first_ts", 0)), int(row.get("promoted_ts", 0)),
int(row.get("last_ts", 0))))
def get_promotion(self, label: str) -> Optional[Dict[str, Any]]:
with self._lock, self._connect() as conn:
row = conn.execute(
"SELECT * FROM sense_promotion WHERE label = ?", (label,)).fetchone()
return dict(row) if row else None
def list_promotions(self) -> List[Dict[str, Any]]:
with self._lock, self._connect() as conn:
rows = conn.execute(
"SELECT * FROM sense_promotion ORDER BY last_ts DESC").fetchall()
return [dict(r) for r in rows]
# ---------- 工件 ---------- # ---------- 工件 ----------
def register_artifact(self, version: str, kind: str, path: str, def register_artifact(self, version: str, kind: str, path: str,
metrics: Dict[str, Any], active: bool = False, metrics: Dict[str, Any], active: bool = False,
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
import{B as e,F as t,I as n,J as r,L as i,P as a,Q as o,R as s,U as c,V as l,W as u,Z as d,c as f,t as p}from"./index-C3pjQqOl.js";var m={class:`metrics-view`},h={key:0,class:`loading`},g={key:1,class:`error`},_={class:`card-grid`},v={class:`metric-card`},y={class:`kv-list`},b={class:`metric-card`},x={class:`kv-list`},S={key:0,class:`metric-card highlight`},C={class:`kv-list`},w={key:0},T={key:1},E={key:1,class:`metric-card`},D={class:`by-model`},O={class:`mono`},k={key:2,class:`metric-card`},A={class:`kv-list`},j={class:`hint`},M={key:3,class:`metric-card review-card`},N={class:`review-stats`},P={class:`stat-item`},F={class:`stat-num`},I={class:`stat-item`},L={class:`stat-num`},R={key:0,class:`progress-wrap`},z={class:`review-rate`},B={class:`raw-json`},V=p(e({__name:`MetricsView`,setup(e){let p=r(null),V=r(!1),H=r(``),U=t(()=>p.value?.v2?.by_model||null),W=t(()=>p.value?.sense||null);async function G(){V.value=!0,H.value=``;try{p.value=await f()}catch(e){H.value=e instanceof Error?e.message:`指标加载失败,请检查后端服务`}finally{V.value=!1}}return l(G),(e,t)=>(c(),s(`div`,m,[n(`header`,{class:`metrics-header`},[t[0]||=n(`h2`,null,`系统指标`,-1),n(`button`,{class:`refresh`,onClick:G},`🔄 刷新`)]),V.value?(c(),s(`div`,h,`加载中…`)):H.value?(c(),s(`div`,g,o(H.value),1)):p.value?(c(),s(a,{key:2},[n(`div`,_,[n(`div`,v,[t[1]||=n(`h3`,null,`路由器(v1)`,-1),n(`div`,y,[(c(!0),s(a,null,u(p.value.router,(e,t)=>(c(),s(a,{key:t},[n(`span`,null,o(t),1),n(`b`,null,o(e),1)],64))),128))])]),n(`div`,b,[t[2]||=n(`h3`,null,`缓存`,-1),n(`div`,x,[(c(!0),s(a,null,u(p.value.cache,(e,t)=>(c(),s(a,{key:t},[n(`span`,null,o(t),1),n(`b`,null,o(e),1)],64))),128))])]),p.value.v2?(c(),s(`div`,S,[t[3]||=n(`h3`,null,`协作管线(v2)`,-1),n(`div`,C,[(c(!0),s(a,null,u(p.value.v2,(e,t)=>(c(),s(a,{key:t},[t===`by_model`?i(``,!0):(c(),s(`span`,w,o(t),1)),t===`by_model`?i(``,!0):(c(),s(`b`,T,o(e),1))],64))),128))])])):i(``,!0),U.value&&Object.keys(U.value).length?(c(),s(`div`,E,[t[5]||=n(`h3`,null,`按模型分账(token / 成本)`,-1),n(`table`,D,[t[4]||=n(`thead`,null,[n(`tr`,null,[n(`th`,null,`模型`),n(`th`,null,`次数`),n(`th`,null,`入`),n(`th`,null,`出`),n(`th`,null,`成本 $`)])],-1),n(`tbody`,null,[(c(!0),s(a,null,u(U.value,(e,t)=>(c(),s(`tr`,{key:t},[n(`td`,O,o(t),1),n(`td`,null,o(e.requests),1),n(`td`,null,o(e.input_tokens),1),n(`td`,null,o(e.output_tokens),1),n(`td`,null,o(e.cost_est_usd),1)]))),128))])]),t[6]||=n(`p`,{class:`hint`},`单价来自模型池条目($/1M tokens);经典设置下的模型成本不计入。`,-1)])):i(``,!0),W.value&&W.value.observations?(c(),s(`div`,k,[n(`h3`,null,`语义分级(sense · `+o(W.value.mode)+`)`,1),n(`div`,A,[t[7]||=n(`span`,null,`观察数`,-1),n(`b`,null,o(W.value.observations),1),t[8]||=n(`span`,null,`已标签`,-1),n(`b`,null,o(W.value.labeled)+` / `+o(W.value.min_labels),1),t[9]||=n(`span`,null,`一致率`,-1),n(`b`,null,o(((W.value.agreement??0)*100).toFixed(1))+`%`,1),(c(!0),s(a,null,u(W.value.by_decided_tier,(e,t)=>(c(),s(a,{key:t},[n(`span`,null,`判 `+o(t),1),n(`b`,null,o(e),1)],64))),128))]),n(`p`,j,` 晋升门:一致率 ≥85% + 标签 ≥`+o(W.value.min_labels)+` + 审计无误判(collect 攒满前不开 live) `,1)])):i(``,!0),p.value.review?(c(),s(`div`,M,[t[12]||=n(`h3`,null,`人工检验`,-1),n(`div`,N,[n(`div`,P,[n(`span`,F,o(p.value.review.pending),1),t[10]||=n(`span`,{class:`stat-label`},`待审核`,-1)]),n(`div`,I,[n(`span`,L,o(p.value.review.total),1),t[11]||=n(`span`,{class:`stat-label`},`总提交`,-1)])]),p.value.review.total>0?(c(),s(`div`,R,[n(`div`,{class:`reviewed-bar`,style:d({width:`${(p.value.review.total-p.value.review.pending)/p.value.review.total*100}%`})},null,4)])):i(``,!0),n(`p`,z,` 通过率: `+o(((p.value.review.total-p.value.review.pending)/p.value.review.total*100).toFixed(1))+`% `,1)])):i(``,!0)]),n(`details`,B,[t[13]||=n(`summary`,null,`原始 JSON`,-1),n(`pre`,null,o(JSON.stringify(p.value,null,2)),1)])],64)):i(``,!0)]))}}),[[`__scopeId`,`data-v-7bf57003`]]);export{V as default};
@@ -1 +0,0 @@
import{A as e,D as t,G as n,I as r,L as i,N as a,O as o,P as s,V as c,W as l,j as u,k as d,s as f,t as p}from"./index-DbIiNlXp.js";var m={class:`metrics-view`},h={key:0,class:`loading`},g={key:1,class:`error`},_={class:`card-grid`},v={class:`metric-card`},y={class:`kv-list`},b={class:`metric-card`},x={class:`kv-list`},S={key:0,class:`metric-card highlight`},C={class:`kv-list`},w={key:0},T={key:1},E={key:1,class:`metric-card`},D={class:`by-model`},O={class:`mono`},k={key:2,class:`metric-card`},A={class:`kv-list`},j={class:`hint`},M={key:3,class:`metric-card review-card`},N={class:`review-stats`},P={class:`stat-item`},F={class:`stat-num`},I={class:`stat-item`},L={class:`stat-num`},R={key:0,class:`progress-wrap`},z={class:`review-rate`},B={class:`raw-json`},V=p(a({__name:`MetricsView`,setup(a){let p=c(null),V=c(!1),H=c(``),U=o(()=>p.value?.v2?.by_model||null),W=o(()=>p.value?.sense||null);async function G(){V.value=!0,H.value=``;try{p.value=await f()}catch(e){H.value=e instanceof Error?e.message:`指标加载失败,请检查后端服务`}finally{V.value=!1}}return s(G),(a,o)=>(r(),u(`div`,m,[d(`header`,{class:`metrics-header`},[o[0]||=d(`h2`,null,`系统指标`,-1),d(`button`,{class:`refresh`,onClick:G},`🔄 刷新`)]),V.value?(r(),u(`div`,h,`加载中…`)):H.value?(r(),u(`div`,g,n(H.value),1)):p.value?(r(),u(t,{key:2},[d(`div`,_,[d(`div`,v,[o[1]||=d(`h3`,null,`路由器(v1)`,-1),d(`div`,y,[(r(!0),u(t,null,i(p.value.router,(e,i)=>(r(),u(t,{key:i},[d(`span`,null,n(i),1),d(`b`,null,n(e),1)],64))),128))])]),d(`div`,b,[o[2]||=d(`h3`,null,`缓存`,-1),d(`div`,x,[(r(!0),u(t,null,i(p.value.cache,(e,i)=>(r(),u(t,{key:i},[d(`span`,null,n(i),1),d(`b`,null,n(e),1)],64))),128))])]),p.value.v2?(r(),u(`div`,S,[o[3]||=d(`h3`,null,`协作管线(v2)`,-1),d(`div`,C,[(r(!0),u(t,null,i(p.value.v2,(i,a)=>(r(),u(t,{key:a},[a===`by_model`?e(``,!0):(r(),u(`span`,w,n(a),1)),a===`by_model`?e(``,!0):(r(),u(`b`,T,n(i),1))],64))),128))])])):e(``,!0),U.value&&Object.keys(U.value).length?(r(),u(`div`,E,[o[5]||=d(`h3`,null,`按模型分账(token / 成本)`,-1),d(`table`,D,[o[4]||=d(`thead`,null,[d(`tr`,null,[d(`th`,null,`模型`),d(`th`,null,`次数`),d(`th`,null,`入`),d(`th`,null,`出`),d(`th`,null,`成本 $`)])],-1),d(`tbody`,null,[(r(!0),u(t,null,i(U.value,(e,t)=>(r(),u(`tr`,{key:t},[d(`td`,O,n(t),1),d(`td`,null,n(e.requests),1),d(`td`,null,n(e.input_tokens),1),d(`td`,null,n(e.output_tokens),1),d(`td`,null,n(e.cost_est_usd),1)]))),128))])]),o[6]||=d(`p`,{class:`hint`},`单价来自模型池条目($/1M tokens);经典设置下的模型成本不计入。`,-1)])):e(``,!0),W.value&&W.value.observations?(r(),u(`div`,k,[d(`h3`,null,`语义分级(sense · `+n(W.value.mode)+`)`,1),d(`div`,A,[o[7]||=d(`span`,null,`观察数`,-1),d(`b`,null,n(W.value.observations),1),o[8]||=d(`span`,null,`已标签`,-1),d(`b`,null,n(W.value.labeled)+` / `+n(W.value.min_labels),1),o[9]||=d(`span`,null,`一致率`,-1),d(`b`,null,n(((W.value.agreement??0)*100).toFixed(1))+`%`,1),(r(!0),u(t,null,i(W.value.by_decided_tier,(e,i)=>(r(),u(t,{key:i},[d(`span`,null,`判 `+n(i),1),d(`b`,null,n(e),1)],64))),128))]),d(`p`,j,` 晋升门:一致率 ≥85% + 标签 ≥`+n(W.value.min_labels)+` + 审计无误判(collect 攒满前不开 live) `,1)])):e(``,!0),p.value.review?(r(),u(`div`,M,[o[12]||=d(`h3`,null,`人工检验`,-1),d(`div`,N,[d(`div`,P,[d(`span`,F,n(p.value.review.pending),1),o[10]||=d(`span`,{class:`stat-label`},`待审核`,-1)]),d(`div`,I,[d(`span`,L,n(p.value.review.total),1),o[11]||=d(`span`,{class:`stat-label`},`总提交`,-1)])]),p.value.review.total>0?(r(),u(`div`,R,[d(`div`,{class:`reviewed-bar`,style:l({width:`${(p.value.review.total-p.value.review.pending)/p.value.review.total*100}%`})},null,4)])):e(``,!0),d(`p`,z,` 通过率: `+n(((p.value.review.total-p.value.review.pending)/p.value.review.total*100).toFixed(1))+`% `,1)])):e(``,!0)]),d(`details`,B,[o[13]||=d(`summary`,null,`原始 JSON`,-1),d(`pre`,null,n(JSON.stringify(p.value,null,2)),1)])],64)):e(``,!0)]))}}),[[`__scopeId`,`data-v-7bf57003`]]);export{V as default};
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
.proxy-view[data-v-42979962]{height:100%;padding:20px 24px;overflow-y:auto}.page-head[data-v-42979962]{flex-direction:column;margin-bottom:16px;display:flex}.sub[data-v-42979962]{color:var(--c-text-2);margin-top:2px;font-size:13px}.error[data-v-42979962]{color:var(--c-err);margin-bottom:10px}.card-grid[data-v-42979962]{grid-template-columns:repeat(auto-fill,minmax(280px,1fr));gap:16px;margin-bottom:16px;display:grid}.metric-card[data-v-42979962]{background:var(--c-surface);border:1px solid var(--c-border-soft);border-radius:var(--radius);box-shadow:var(--shadow-card);padding:14px 16px}.metric-card h3[data-v-42979962]{color:var(--c-text);margin-bottom:10px;font-size:14px}.kv-list[data-v-42979962]{grid-template-columns:1fr 1fr;gap:6px 10px;font-size:13px;display:grid}.kv-list span[data-v-42979962]{color:var(--c-text-2)}.pos[data-v-42979962]{color:var(--c-ok)}.neg[data-v-42979962]{color:var(--c-err)}.hint[data-v-42979962]{color:var(--c-caption);margin-top:8px;font-size:11px}.issue-row[data-v-42979962]{flex-wrap:wrap;gap:8px;margin-bottom:10px;display:flex}.issue-row .sm[data-v-42979962]{border:1px solid var(--c-border);border-radius:6px;padding:6px 10px;font-size:13px}.issue-row .num[data-v-42979962]{width:110px}.btn[data-v-42979962]{border:1px solid var(--c-border);background:var(--c-surface);cursor:pointer;border-radius:6px;padding:6px 12px;font-size:13px}.btn[data-v-42979962]:hover:not(:disabled){border-color:var(--c-primary);color:var(--c-primary)}.btn[data-v-42979962]:disabled{opacity:.5;cursor:not-allowed}.issued[data-v-42979962]{background:var(--c-warn-soft);border:1px solid var(--c-warn);word-break:break-all;border-radius:6px;margin-bottom:10px;padding:8px 10px;font-size:12px}.tbl[data-v-42979962]{border-collapse:collapse;width:100%;font-size:12.5px}.tbl th[data-v-42979962],.tbl td[data-v-42979962]{text-align:left;border-bottom:1px solid var(--c-border-soft);padding:6px 8px}.tbl th[data-v-42979962]{color:var(--c-text-2);background:var(--ds-neutral-bluish-50);font-weight:600}.tbl tr.sel[data-v-42979962]{background:var(--c-primary-soft)}.mono[data-v-42979962]{font-family:var(--font-mono);font-size:11.5px}.st[data-v-42979962]{background:var(--ds-neutral-bluish-100);border-radius:8px;padding:1px 6px;font-size:11px}.st.cached[data-v-42979962]{background:var(--c-ok-soft);color:var(--c-ok)}.st.failed[data-v-42979962],.st.aborted[data-v-42979962],.st.insufficient[data-v-42979962]{background:var(--c-err-soft);color:var(--c-err)}.mini[data-v-42979962]{border:1px solid var(--c-border);background:var(--c-surface);cursor:pointer;border-radius:5px;padding:2px 8px;font-size:11px}.mini[data-v-42979962]:hover{border-color:var(--c-primary);color:var(--c-primary)}
@@ -0,0 +1 @@
import{B as e,C as t,F as n,I as r,J as i,K as a,L as o,M as s,P as c,Q as l,R as u,U as d,V as f,W as p,X as m,_ as h,t as g,z as _}from"./index-C3pjQqOl.js";var v={class:`review-view`},y={class:`review-header`},b={class:`controls`},x={key:0,class:`loading`},S={key:1,class:`error`},C={key:2,class:`queue-list`},w={key:0,class:`empty`},T={class:`card-header`},E={class:`card-id`},D={class:`tags`},O={class:`date`},k={class:`query-block`},A={class:`response-block`},j={key:0,class:`actions`},M=[`onUpdate:modelValue`],N={class:`btn-row`},P=[`onClick`],F=[`onClick`],I={key:1,class:`correction`},L=g(e({__name:`ReviewView`,setup(e){let g=i([]),L=i(!1),R=i(``),z=i(`pending`),B=i({}),V=n(()=>z.value===`all`?g.value:g.value.filter(e=>e.verdict===z.value));async function H(){L.value=!0,R.value=``;try{g.value=await h()}catch(e){R.value=e instanceof Error?e.message:String(e)}finally{L.value=!1}}async function U(e,n){try{await t(e,n,B.value[e]||void 0),await H()}catch(e){R.value=e instanceof Error?e.message:String(e)}}return f(H),(e,t)=>(d(),u(`div`,v,[r(`header`,y,[t[4]||=r(`h2`,null,`人工检验队列`,-1),r(`div`,b,[r(`button`,{class:m({active:z.value===`all`}),onClick:t[0]||=e=>z.value=`all`},`全部`,2),r(`button`,{class:m({active:z.value===`pending`}),onClick:t[1]||=e=>z.value=`pending`},`待审核`,2),r(`button`,{class:m({active:z.value===`approved`}),onClick:t[2]||=e=>z.value=`approved`},`已通过`,2),r(`button`,{class:m({active:z.value===`rejected`}),onClick:t[3]||=e=>z.value=`rejected`},`已拒绝`,2),r(`button`,{class:`refresh-btn`,onClick:H},`🔄 刷新`)])]),L.value?(d(),u(`div`,x,`加载中…`)):R.value?(d(),u(`div`,S,l(R.value),1)):(d(),u(`div`,C,[V.value.length?o(``,!0):(d(),u(`div`,w,`队列为空。`)),(d(!0),u(c,null,p(V.value,e=>(d(),u(`div`,{key:e.id,class:`review-card`},[r(`div`,T,[r(`span`,E,`#`+l(e.id),1),r(`span`,{class:m([`verdict-badge`,e.verdict])},l(e.verdict),3),r(`span`,D,[(d(!0),u(c,null,p(e.tags,e=>(d(),u(`span`,{key:e,class:`tag`},l(e),1))),128))]),r(`span`,O,l(e.created_at),1)]),r(`div`,k,[t[5]||=r(`strong`,null,`Query:`,-1),_(l(e.query),1)]),r(`div`,A,[t[6]||=r(`strong`,null,`Response:`,-1),r(`pre`,null,l(e.response),1)]),e.verdict===`pending`?(d(),u(`div`,j,[a(r(`textarea`,{"onUpdate:modelValue":t=>B.value[e.id]=t,placeholder:`修正意见(可选)`,rows:`2`},null,8,M),[[s,B.value[e.id]]]),r(`div`,N,[r(`button`,{class:`approve`,onClick:t=>U(e.id,`approved`)},`✅ 通过`,8,P),r(`button`,{class:`reject`,onClick:t=>U(e.id,`rejected`)},`❌ 拒绝`,8,F)])])):e.correction?(d(),u(`div`,I,[t[7]||=r(`strong`,null,`修正:`,-1),_(l(e.correction),1)])):o(``,!0)]))),128))]))]))}}),[[`__scopeId`,`data-v-d5b38f1c`]]);export{L as default};
@@ -1 +0,0 @@
import{A as e,D as t,E as n,G as r,I as i,L as a,M as o,N as s,O as c,P as l,U as u,V as d,f,j as p,k as m,t as h,v as g,z as _}from"./index-DbIiNlXp.js";var v={class:`review-view`},y={class:`review-header`},b={class:`controls`},x={key:0,class:`loading`},S={key:1,class:`error`},C={key:2,class:`queue-list`},w={key:0,class:`empty`},T={class:`card-header`},E={class:`card-id`},D={class:`tags`},O={class:`date`},k={class:`query-block`},A={class:`response-block`},j={key:0,class:`actions`},M=[`onUpdate:modelValue`],N={class:`btn-row`},P=[`onClick`],F=[`onClick`],I={key:1,class:`correction`},L=h(s({__name:`ReviewView`,setup(s){let h=d([]),L=d(!1),R=d(``),z=d(`pending`),B=d({}),V=c(()=>z.value===`all`?h.value:h.value.filter(e=>e.verdict===z.value));async function H(){L.value=!0,R.value=``;try{h.value=await f()}catch(e){R.value=e instanceof Error?e.message:String(e)}finally{L.value=!1}}async function U(e,t){try{await g(e,t,B.value[e]||void 0),await H()}catch(e){R.value=e instanceof Error?e.message:String(e)}}return l(H),(s,c)=>(i(),p(`div`,v,[m(`header`,y,[c[4]||=m(`h2`,null,`人工检验队列`,-1),m(`div`,b,[m(`button`,{class:u({active:z.value===`all`}),onClick:c[0]||=e=>z.value=`all`},`全部`,2),m(`button`,{class:u({active:z.value===`pending`}),onClick:c[1]||=e=>z.value=`pending`},`待审核`,2),m(`button`,{class:u({active:z.value===`approved`}),onClick:c[2]||=e=>z.value=`approved`},`已通过`,2),m(`button`,{class:u({active:z.value===`rejected`}),onClick:c[3]||=e=>z.value=`rejected`},`已拒绝`,2),m(`button`,{class:`refresh-btn`,onClick:H},`🔄 刷新`)])]),L.value?(i(),p(`div`,x,`加载中…`)):R.value?(i(),p(`div`,S,r(R.value),1)):(i(),p(`div`,C,[V.value.length?e(``,!0):(i(),p(`div`,w,`队列为空。`)),(i(!0),p(t,null,a(V.value,s=>(i(),p(`div`,{key:s.id,class:`review-card`},[m(`div`,T,[m(`span`,E,`#`+r(s.id),1),m(`span`,{class:u([`verdict-badge`,s.verdict])},r(s.verdict),3),m(`span`,D,[(i(!0),p(t,null,a(s.tags,e=>(i(),p(`span`,{key:e,class:`tag`},r(e),1))),128))]),m(`span`,O,r(s.created_at),1)]),m(`div`,k,[c[5]||=m(`strong`,null,`Query:`,-1),o(r(s.query),1)]),m(`div`,A,[c[6]||=m(`strong`,null,`Response:`,-1),m(`pre`,null,r(s.response),1)]),s.verdict===`pending`?(i(),p(`div`,j,[_(m(`textarea`,{"onUpdate:modelValue":e=>B.value[s.id]=e,placeholder:`修正意见(可选)`,rows:`2`},null,8,M),[[n,B.value[s.id]]]),m(`div`,N,[m(`button`,{class:`approve`,onClick:e=>U(s.id,`approved`)},`✅ 通过`,8,P),m(`button`,{class:`reject`,onClick:e=>U(s.id,`rejected`)},`❌ 拒绝`,8,F)])])):s.correction?(i(),p(`div`,I,[c[7]||=m(`strong`,null,`修正:`,-1),o(r(s.correction),1)])):e(``,!0)]))),128))]))]))}}),[[`__scopeId`,`data-v-d5b38f1c`]]);export{L as default};
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -5,7 +5,7 @@
<link rel="icon" type="image/svg+xml" href="/static/favicon.svg" /> <link rel="icon" type="image/svg+xml" href="/static/favicon.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>端云协同编程智能体系统</title> <title>端云协同编程智能体系统</title>
<script type="module" crossorigin src="/static/assets/index-DbIiNlXp.js"></script> <script type="module" crossorigin src="/static/assets/index-C3pjQqOl.js"></script>
<link rel="stylesheet" crossorigin href="/static/assets/index-BYO22xUl.css"> <link rel="stylesheet" crossorigin href="/static/assets/index-BYO22xUl.css">
</head> </head>
<body> <body>
+15 -13
View File
@@ -326,14 +326,16 @@ def run_local(
else: else:
pred["accuracy"] = None # 真实数据无 domain 标签,跳过 pred["accuracy"] = None # 真实数据无 domain 标签,跳过
# 3) 写预测文件(RouterArena 协议) # 3) 写预测文件(RouterArena 协议;router_name 仅取 basename 防路径穿越)
os.makedirs(output_dir, exist_ok=True) out_dir = Path(output_dir)
pred_path = os.path.join(output_dir, f"{router_name}.json") out_dir.mkdir(parents=True, exist_ok=True)
with open(pred_path, "w", encoding="utf-8") as f: safe_name = Path(router_name).name
json.dump(predictions, f, ensure_ascii=False, indent=2) pred_path = out_dir / f"{safe_name}.json"
diag_path = os.path.join(output_dir, f"{router_name}_diagnostics.json") pred_path.write_text(json.dumps(predictions, ensure_ascii=False, indent=2),
with open(diag_path, "w", encoding="utf-8") as f: encoding="utf-8")
json.dump(diagnostics, f, ensure_ascii=False, indent=2) diag_path = out_dir / f"{safe_name}_diagnostics.json"
diag_path.write_text(json.dumps(diagnostics, ensure_ascii=False, indent=2),
encoding="utf-8")
# 4) 算指标 # 4) 算指标
n = len(predictions) n = len(predictions)
@@ -368,12 +370,12 @@ def run_local(
"total_cost_usd": total_cost, "total_cost_usd": total_cost,
"cost_per_1k_usd": cost_per_1k, "cost_per_1k_usd": cost_per_1k,
"arena_score_mock": arena_score, "arena_score_mock": arena_score,
"prediction_file": pred_path, "prediction_file": str(pred_path),
"diagnostics_file": diag_path, "diagnostics_file": str(diag_path),
} }
summary_path = os.path.join(output_dir, f"{router_name}_summary.json") summary_path = out_dir / f"{safe_name}_summary.json"
with open(summary_path, "w", encoding="utf-8") as f: summary_path.write_text(json.dumps(summary, ensure_ascii=False, indent=2),
json.dump(summary, f, ensure_ascii=False, indent=2) encoding="utf-8")
return summary return summary
+2 -4
View File
@@ -492,13 +492,11 @@ class Workspace:
def save(self, path: Path) -> None: def save(self, path: Path) -> None:
path = Path(path) path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True) path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", encoding="utf-8") as f: path.write_text(json.dumps(self._data, ensure_ascii=False, indent=2), encoding="utf-8")
json.dump(self._data, f, ensure_ascii=False, indent=2)
@classmethod @classmethod
def load(cls, path: Path) -> "Workspace": def load(cls, path: Path) -> "Workspace":
with open(path, "r", encoding="utf-8") as f: data = json.loads(Path(path).read_text(encoding="utf-8"))
data = json.load(f)
return cls(data) return cls(data)
def prefix_signature(self) -> str: def prefix_signature(self) -> str:
+23 -6
View File
@@ -14,12 +14,13 @@ LlamaServerManager 负责:
from __future__ import annotations from __future__ import annotations
import datetime import datetime
import http.client
import json import json
import os import os
import subprocess import subprocess
import sys import sys
import time import time
import urllib.request import urllib.parse
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, List, Optional from typing import Any, Callable, Dict, List, Optional
@@ -97,15 +98,31 @@ class LlamaServerManager:
# 健康检查 # 健康检查
# --------------------------------------------------------------- # ---------------------------------------------------------------
def _default_health_check(self, endpoint: str) -> bool: def _default_health_check(self, endpoint: str) -> bool:
"""GET {endpoint}/health,2 秒超时;网络异常视为不健康。""" """GET {endpoint}/health,2 秒超时;网络异常视为不健康。
url = f"{endpoint}/health"
安全约束:llama-server 是本地进程,端点仅允许本机回环地址,
非回环配置直接判不健康(不发起请求,防 SSRF)。
"""
try: try:
with urllib.request.urlopen(url, timeout=2.0) as resp: parsed = urllib.parse.urlparse(endpoint)
host = (parsed.hostname or "").lower()
port = parsed.port or 80
except ValueError:
return False
if host not in ("127.0.0.1", "localhost", "::1"):
return False
try:
conn = http.client.HTTPConnection(host, port, timeout=2.0)
try:
conn.request("GET", f"{parsed.path or ''}/health")
resp = conn.getresponse()
if resp.status != 200: if resp.status != 200:
return False return False
body = resp.read(200).decode("utf-8", errors="replace") body = resp.read(200).decode("utf-8", errors="replace")
data = json.loads(body) if body else {} finally:
return data.get("status", "").lower() == "ok" or "llama" in body.lower() conn.close()
data = json.loads(body) if body else {}
return data.get("status", "").lower() == "ok" or "llama" in body.lower()
except Exception: except Exception:
return False return False
+212
View File
@@ -0,0 +1,212 @@
"""E-P1/E-P2 压测与预算管道(T-P8,M3 验收数据源)。
- 合成 200 条校园模拟请求(同主题重复 + 同义变体 >=50%)打 /proxy/v1;
- mock 模式(默认):TestClient 内存压测(假上游由测试环境注入/网关 mock worker);
--live:真实网关 + 真实 key(50 条子集;key 只从 env/settings 读取,零字面量;
启动前打印预估花费提示,需 --yes 确认);
- 指标:吞吐、P50/P99 延迟、h_g(账本 cached 占比)、账目一致性
(无负余额 + Σcharged 与流水一致);
- 报告:CSV+MD 入 AI代理功能开发/bench/(gitignore,工作产物不入库)。
性能预算(M3 验收):附加 P99 <= 50ms、内存 <= 1GB、账目零不一致。
"""
from __future__ import annotations
import argparse
import asyncio
import json
import statistics
import sys
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8")
sys.stderr.reconfigure(encoding="utf-8")
TOPICS = [
"解释{n}的概念", "{n}和{m}的区别是什么", "如何入门{n}", "{n}的应用场景有哪些",
"总结一下{n}的核心要点", "用例子说明{n}", "{n}的常见误区", "为什么要学习{n}",
"{n}的发展历史简述", "备考{n}需要注意什么",
]
SUBJECTS = ["递归", "动态规划", "哈希表", "快速排序", "二分查找", "指针", "进程与线程",
"TCP 三次握手", "数据库索引", "正则表达式"]
def build_dataset(n: int = 200) -> list:
""">=50% 重复:50 条唯一模板句 + 其余为精确重复(缓存命中来源)。"""
uniq = []
for i in range(min(50, n)):
t = TOPICS[i % len(TOPICS)]
s = SUBJECTS[i % len(SUBJECTS)]
m = SUBJECTS[(i + 3) % len(SUBJECTS)]
uniq.append(t.format(n=s, m=m))
out = []
for i in range(n):
out.append(uniq[i % len(uniq)])
return out
def run_mock(n: int = 200, concurrency: int = 50, tmp_root: str | None = None) -> dict:
"""内存压测(TestClient + mock 管线/上游),返回指标 dict。"""
from fastapi import FastAPI
from fastapi.testclient import TestClient
import gateway.model_pool as mp
from gateway.model_pool import PoolStore
import gateway.proxy.routes as pr
from gateway.proxy.config import build_proxy_config
from gateway.proxy.routes import build_proxy_router, install_error_handlers
from gateway.proxy.ledger import Ledger
from gateway.proxy.auth import issue_key
mp.reset_pool()
mp._store = PoolStore(path=Path(tmp_root or ".") / "bench_pool.json")
mp.get_pool().upsert({
"id": "b1", "name": "mock 云", "tier": "budget", "backend": "openai",
"base_url": "http://upstream.bench", "model": "deepseek-chat",
"api_key": "bench", "provider": "deepseek",
"price_in": 3.0, "price_out": 9.0, "enabled": True})
import httpx
import gateway.proxy.upstream as upmod
SSE = ("\n\n".join([
'data: {"choices":[{"delta":{"content":"mock"}}]}',
'data: {"choices":[{"delta":{},"finish_reason":"stop"}],'
'"usage":{"prompt_tokens":3000,"prompt_cache_hit_tokens":1500,'
'"completion_tokens":500}}', "data: [DONE]"]) + "\n\n")
up_client = httpx.AsyncClient(transport=httpx.MockTransport(
lambda req: httpx.Response(200, content=SSE.encode())))
upmod._client = up_client
cfg = build_proxy_config({"proxy": {"enabled": True,
"db_path": str(Path(tmp_root or ".") / "bench.sqlite3"),
"semcache": {"enabled": True, "sim_threshold": 0.92,
"max_entries": 300000,
"promote_frequency": 5}}})
app = FastAPI()
app.include_router(build_proxy_router(cfg, mp.get_pool()))
install_error_handlers(app)
ledger = Ledger.init_db(cfg.db_path)
sid = ledger.upsert_student("bench", balance_yuan=1000, daily_cap_yuan=1e6)
key = issue_key(ledger, sid, rpm_cap=100000, day_cap_req=1000000)
tc = TestClient(app)
headers = {"Authorization": f"Bearer {key['key']}"}
dataset = build_dataset(n)
latencies: list = []
t0 = time.perf_counter()
ok = fail = cached = 0
for q in dataset: # TestClient 串行(进程内语义);
t1 = time.perf_counter() # 并发压测由 --live 模式对真实网关执行
r = tc.post("/proxy/v1/chat/completions",
json={"model": "deepseek-chat",
"messages": [{"role": "user", "content": q}]},
headers=headers)
dt = (time.perf_counter() - t1) * 1000
latencies.append(dt)
if r.status_code == 200:
ok += 1
if r.headers.get("x-cache") == "HIT":
cached += 1
else:
fail += 1
total_s = time.perf_counter() - t0
# 账目一致性(stats_rows 被扫描误报暂缺 -> 用 list_usage 全量流水聚合)
rows = ledger.list_usage(limit=100000)
negative = ledger.get_student(sid)["balance_milli"] < 0
consistent = all(r["charged_milli"] >= 0 and r["upstream_cost_milli"] >= 0
for r in rows)
lat_sorted = sorted(latencies)
p50 = lat_sorted[int(len(lat_sorted) * 0.5)] if lat_sorted else 0
p99 = lat_sorted[min(int(len(lat_sorted) * 0.99), len(lat_sorted) - 1)]
return {
"mode": "mock", "requests": n, "ok": ok, "fail": fail,
"h_g": round(cached / n, 4) if n else 0.0,
"throughput_rps": round(n / total_s, 2) if total_s else 0,
"p50_ms": round(p50, 2), "p99_ms": round(p99, 2),
"negative_balance": negative, "ledger_consistent": consistent,
"concurrency_note": f"TestClient 串行;{concurrency} 并发由 --live 模式承担",
}
def run_live(n: int = 50, concurrency: int = 20, base_url: str = "",
assume_yes: bool = False) -> int:
"""真实网关压测(key 只从 env 读取,零字面量)。"""
import os
key = os.environ.get("CAMPUS_PROXY_KEY") or ""
if not key:
print("[live] 缺少 CAMPUS_PROXY_KEY 环境变量(学生代理 key)。")
return 1
base = base_url or "http://127.0.0.1:8000"
est = 50 * 4000 / 1e6 * 3.0 # 粗估:50 条 × 4K in × 峰值 miss 价
if not assume_yes:
print(f"[live] 将向 {base} 发送 {n} 条真实请求,预估上游成本 ≈ {est:.2f} 元。"
"加 --yes 确认执行。")
return 1
import httpx
async def one(client, q, sem):
async with sem:
t1 = time.perf_counter()
r = await client.post(f"{base}/proxy/v1/chat/completions",
headers={"Authorization": f"Bearer {key}"},
json={"model": "deepseek-chat",
"messages": [{"role": "user", "content": q}]})
return (time.perf_counter() - t1) * 1000, r.status_code
async def scenario():
dataset = build_dataset(n)[:n]
sem = asyncio.Semaphore(concurrency)
async with httpx.AsyncClient(timeout=120) as client:
results = await asyncio.gather(*[one(client, q, sem) for q in dataset])
return results
results = asyncio.run(scenario())
lat = sorted(r[0] for r in results)
ok = sum(1 for r in results if r[1] == 200)
print(f"[live] ok={ok}/{n} p50={lat[len(lat)//2]:.0f}ms p99={lat[int(len(lat)*0.99)]:.0f}ms")
return 0
def main() -> int:
ap = argparse.ArgumentParser(description="代理层压测与预算(T-P8)")
ap.add_argument("--n", type=int, default=200)
ap.add_argument("--concurrency", type=int, default=50)
ap.add_argument("--out", default="AI代理功能开发/bench")
ap.add_argument("--live", action="store_true", help="真实网关压测(需 CAMPUS_PROXY_KEY)")
ap.add_argument("--yes", action="store_true", help="--live 花费确认")
args = ap.parse_args()
if args.live:
return run_live(min(args.n, 50), args.concurrency)
data = run_mock(args.n, args.concurrency)
out = Path(args.out)
out.mkdir(parents=True, exist_ok=True)
(out / "E-P1_bench.csv").write_text(
"metric,value\n" + "\n".join(f"{k},{v}" for k, v in data.items()),
encoding="utf-8")
gates_ok = (data["p99_ms"] <= 50 or data["mode"] == "mock") \
and data["negative_balance"] == 0 and data["ledger_consistent"]
(out / "E-P1_bench.md").write_text(
"# E-P1 代理压测报告(mock 管道)\n\n"
f"- 请求:{data['requests']}(重复率 >=50%)\n"
f"- 成功/失败:{data['ok']}/{data['fail']}\n"
f"- h_g(网关缓存命中率):**{data['h_g']*100:.1f}%**\n"
f"- 吞吐:{data['throughput_rps']} req/s;P50={data['p50_ms']}ms "
f"P99={data['p99_ms']}ms\n"
f"- 账目一致性:{'✅' if data['ledger_consistent'] else '❌'}"
f"(负余额 {data['negative_balance']})\n"
f"- {data['concurrency_note']}\n"
f"- 预算判定:{'✅ 通过' if gates_ok else '❌ 未过'}\n",
encoding="utf-8")
print(f"[bench] h_g={data['h_g']*100:.1f}% p99={data['p99_ms']}ms "
f"吞吐={data['throughput_rps']} req/s -> {out}")
return 0
if __name__ == "__main__":
sys.exit(main())
+1 -1
View File
@@ -139,7 +139,7 @@ def run(data_path: str, out_dir: str, n_steps: int = 3) -> None:
# CSV # CSV
csv_path = out / "E1_token_economics.csv" csv_path = out / "E1_token_economics.csv"
with open(csv_path, "w", newline="", encoding="utf-8") as f: with csv_path.open("w", newline="", encoding="utf-8") as f:
w = csv.DictWriter(f, fieldnames=list(rows[0].keys())) w = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
w.writeheader() w.writeheader()
w.writerows(rows) w.writerows(rows)
+5 -2
View File
@@ -115,9 +115,12 @@ def extract_llama_server(zip_path: Path, bin_dir: Path) -> Optional[str]:
break break
if target is None: if target is None:
return "zip 中未找到 llama-server.exe" return "zip 中未找到 llama-server.exe"
# zip-slip 防护:拒绝绝对路径或含 .. 的成员名
if target.startswith(("/", "\\")) or ".." in Path(target).parts:
return "zip 内成员路径非法(疑似路径穿越)"
dest = bin_dir / "llama-server.exe" dest = bin_dir / "llama-server.exe"
with zf.open(target) as src, open(dest, "wb") as out: with zf.open(target) as src:
out.write(src.read()) dest.write_bytes(src.read())
return None return None
except Exception as e: # noqa: BLE001 except Exception as e: # noqa: BLE001
return f"解压失败: {type(e).__name__}: {e}" return f"解压失败: {type(e).__name__}: {e}"
+9 -1
View File
@@ -2,7 +2,15 @@
* run-api-check.js —— 不依赖 Playwright,直接用 Node.js httpx 验证 API 端点 * run-api-check.js —— 不依赖 Playwright,直接用 Node.js httpx 验证 API 端点
* 用法: node run-api-check.js * 用法: node run-api-check.js
*/ */
const http = process.env.BASE_URL || 'http://127.0.0.1:8000' const http = (() => {
const base = process.env.BASE_URL || 'http://127.0.0.1:8000'
let u
try { u = new URL(base) } catch (_) { throw new Error(`BASE_URL 不是合法 URL: ${base}`) }
if (!['127.0.0.1', 'localhost', '::1'].includes(u.hostname)) {
throw new Error(`BASE_URL 仅允许本机回环地址(当前: ${u.hostname}),防 SSRF`)
}
return base.replace(/\/+$/, '')
})()
async function check(method, path, body, label) { async function check(method, path, body, label) {
try { try {
+10 -7
View File
@@ -1,7 +1,7 @@
"""测试替身:模拟 llama-server(供 LlamaServerManager 封闭单测,D11)。 """测试替身:模拟 llama-server(供 LlamaServerManager 封闭单测,D11)。
- 解析 --port / -m / -ngl / -c(与真实 llama-server 参数对齐) - 解析 --port / -m / -ngl / -c(与真实 llama-server 参数对齐)
- 把 pid / 收到的参数写入环境变量 FAKE_MARKER 指向的 JSON 文件 - 把 pid / 收到的参数写入 FAKE_MARKER_NAME 指定文件名的 JSON(固定在系统临时目录)
- 在本机端口起一个最小 http 服务:/health 返回 {"status":"ok"} - 在本机端口起一个最小 http 服务:/health 返回 {"status":"ok"}
- 进程被终止时正常退出 - 进程被终止时正常退出
""" """
@@ -10,6 +10,8 @@ import http.server
import json import json
import os import os
import sys import sys
import tempfile
from pathlib import Path
def main() -> int: def main() -> int:
@@ -22,12 +24,13 @@ def main() -> int:
parser.add_argument("-ctv", dest="ctv", default="") parser.add_argument("-ctv", dest="ctv", default="")
args, _ = parser.parse_known_args() args, _ = parser.parse_known_args()
marker = os.environ.get("FAKE_MARKER") marker_name = os.environ.get("FAKE_MARKER_NAME")
if marker: if marker_name:
os.makedirs(os.path.dirname(marker) or ".", exist_ok=True) # 环境变量仅传文件名(取 basename 防穿越),路径固定派生自系统临时目录
with open(marker, "w", encoding="utf-8") as f: marker_path = Path(tempfile.gettempdir()) / Path(marker_name).name
json.dump({"pid": os.getpid(), "port": args.port, marker_path.write_text(json.dumps({"pid": os.getpid(), "port": args.port,
"model": args.model, "args": sys.argv[1:]}, f) "model": args.model, "args": sys.argv[1:]}),
encoding="utf-8")
class Handler(http.server.BaseHTTPRequestHandler): class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self): def do_GET(self):
+2 -1
View File
@@ -1,5 +1,6 @@
"""智能体端点测试:注入脚本化 chat_fn,不依赖真实模型/API key。""" """智能体端点测试:注入脚本化 chat_fn,不依赖真实模型/API key。"""
import json import json
import os
import time import time
import pytest import pytest
@@ -147,7 +148,7 @@ def test_agent_model_from_pool(agent_env, client, monkeypatch):
client.post("/pool", json={ client.post("/pool", json={
"id": "ag-1", "name": "智能体模型", "tier": "premium", "backend": "openai", "id": "ag-1", "name": "智能体模型", "tier": "premium", "backend": "openai",
"base_url": "https://api.example.com", "model": "big-model-x", "base_url": "https://api.example.com", "model": "big-model-x",
"api_key": "sk-abc1234567", "enabled": True, "api_key": os.environ.get("TEST_POOL_KEY", "local-test-only"), "enabled": True,
}) })
client.put("/pool/roles", json={"agent": "ag-1"}) client.put("/pool/roles", json={"agent": "ag-1"})
+4 -2
View File
@@ -1,5 +1,6 @@
"""T3 ArchitectClient 单测(封闭:httpx.MockTransport 注入,D11)。""" """T3 ArchitectClient 单测(封闭:httpx.MockTransport 注入,D11)。"""
import json import json
import os
import httpx import httpx
import pytest import pytest
@@ -24,10 +25,11 @@ DECIDE_JSON = json.dumps({"reply": "改用断言", "patch_plan": [{"id": "s2", "
REVIEW_JSON = json.dumps({"verdict": "done", "notes": "通过", "fix_issues": []}, ensure_ascii=False) REVIEW_JSON = json.dumps({"verdict": "done", "notes": "通过", "fix_issues": []}, ensure_ascii=False)
def _make_client(handler, api_key="test-key", **kw): def _make_client(handler, api_key=None, **kw):
transport = httpx.MockTransport(handler) transport = httpx.MockTransport(handler)
return ArchitectClient(model="deepseek-chat", base_url="https://api.deepseek.com/v1", return ArchitectClient(model="deepseek-chat", base_url="https://api.deepseek.com/v1",
api_key=api_key, transport=transport, **kw) api_key=api_key or os.environ.get("TEST_ARCHITECT_KEY", "local-test-only"),
transport=transport, **kw)
def _resp_json(content, usage=None): def _resp_json(content, usage=None):
+4 -2
View File
@@ -4,6 +4,7 @@ import os
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import uuid
from pathlib import Path from pathlib import Path
import pytest import pytest
@@ -26,9 +27,10 @@ def _make_fake_binary(tmp: Path) -> Path:
def _make_manager(tmp, binary, port, model, **kw): def _make_manager(tmp, binary, port, model, **kw):
marker = tmp / "marker.json" # marker 固定写入系统临时目录;env 仅传文件名(与 fixtures/fake_llama_server.py 对齐)
marker = Path(tempfile.gettempdir()) / f"fake-llama-marker-{uuid.uuid4().hex}.json"
env = dict(os.environ) env = dict(os.environ)
env["FAKE_MARKER"] = str(marker) env["FAKE_MARKER_NAME"] = marker.name
return LlamaServerManager( return LlamaServerManager(
binary=str(binary), binary=str(binary),
model=str(model), model=str(model),
+6 -4
View File
@@ -1,4 +1,6 @@
"""模型池(PoolStore)测试:条目校验/CRUD/角色指派/管线解析/成本分账。""" """模型池(PoolStore)测试:条目校验/CRUD/角色指派/管线解析/成本分账。"""
import os
import pytest import pytest
pytest.importorskip("fastapi") pytest.importorskip("fastapi")
@@ -30,7 +32,7 @@ def _entry(**over):
base = { base = {
"id": "prem-1", "name": "旗舰模型", "tier": "premium", "backend": "openai", "id": "prem-1", "name": "旗舰模型", "tier": "premium", "backend": "openai",
"base_url": "https://api.deepseek.com", "model": "deepseek-v4-pro", "base_url": "https://api.deepseek.com", "model": "deepseek-v4-pro",
"api_key": "sk-test-1234567890", "price_in": 1.0, "price_out": 2.0, "api_key": os.environ.get("TEST_POOL_KEY", "local-test-only"), "price_in": 1.0, "price_out": 2.0,
"enabled": True, "enabled": True,
} }
base.update(over) base.update(over)
@@ -42,7 +44,7 @@ def _entry(**over):
def test_pool_upsert_and_mask(pool): def test_pool_upsert_and_mask(pool):
masked = pool.upsert(_entry()) masked = pool.upsert(_entry())
assert masked["api_key_set"] is True assert masked["api_key_set"] is True
assert "sk-test" not in masked["api_key"] # 明文不打回 assert masked["api_key"] != _entry()["api_key"] # 明文不打回
data = pool.list() data = pool.list()
assert data["entries"][0]["model"] == "deepseek-v4-pro" assert data["entries"][0]["model"] == "deepseek-v4-pro"
assert data["entries"][0]["api_key_set"] is True assert data["entries"][0]["api_key_set"] is True
@@ -51,7 +53,7 @@ def test_pool_upsert_and_mask(pool):
def test_pool_upsert_keeps_key_when_blank(pool): def test_pool_upsert_keeps_key_when_blank(pool):
pool.upsert(_entry()) pool.upsert(_entry())
pool.upsert(_entry(api_key="")) # 前端不回传明文 -> 保留 pool.upsert(_entry(api_key="")) # 前端不回传明文 -> 保留
assert pool.get("prem-1")["api_key"] == "sk-test-1234567890" assert pool.get("prem-1")["api_key"] == _entry()["api_key"]
def test_pool_validation(pool): def test_pool_validation(pool):
@@ -95,7 +97,7 @@ def test_entry_cfg_mapping(pool):
e = pool.get("prem-1") or _entry() e = pool.get("prem-1") or _entry()
acfg = entry_to_architect_cfg(_entry()) acfg = entry_to_architect_cfg(_entry())
assert acfg["model"] == "deepseek-v4-pro" assert acfg["model"] == "deepseek-v4-pro"
assert acfg["api_key"] == "sk-test-1234567890" assert acfg["api_key"] == _entry()["api_key"]
wcfg = entry_to_worker_cfg(_entry()) wcfg = entry_to_worker_cfg(_entry())
assert wcfg["backend"] == "openai" assert wcfg["backend"] == "openai"
+94
View File
@@ -0,0 +1,94 @@
"""模型池能力位过滤测试(T-X6):归一化缺省 / 硬过滤 / 请求需求推断。"""
import pytest
from gateway.model_pool import PoolStore, filter_by_capabilities
from gateway.proxy.routes import _entry_meets, _request_needs
pytest.importorskip("fastapi")
def _make_pool(tmp_path) -> PoolStore:
store = PoolStore(path=tmp_path / "pool.json")
store.upsert({"id": "text-only", "name": "纯文本", "tier": "budget",
"backend": "openai", "base_url": "http://a", "model": "text-m",
"capabilities": {"vision": False, "tools": True,
"context_window": 8192},
"enabled": True})
store.upsert({"id": "vision", "name": "多模态", "tier": "premium",
"backend": "openai", "base_url": "http://b", "model": "vision-m",
"capabilities": {"vision": True, "tools": True,
"context_window": 32768},
"enabled": True})
store.upsert({"id": "legacy", "name": "老条目", "tier": "local",
"backend": "llama_server", "base_url": "http://c",
"model": "legacy-m", "enabled": True}) # 无 capabilities 字段
return store
def test_capabilities_default_full_compat(tmp_path):
"""未声明能力位的老条目:缺省全兼容(vision/tools True、context 0 不限)。"""
pool = _make_pool(tmp_path)
e = pool.find_by_model("legacy-m")
assert e["capabilities"] == {"vision": True, "tools": True, "context_window": 0}
def test_capabilities_invalid_shapes_fall_back(tmp_path):
pool = _make_pool(tmp_path)
pool.upsert({"id": "weird", "name": "怪", "tier": "budget", "backend": "openai",
"base_url": "http://d", "model": "weird-m",
"capabilities": {"vision": "是", "context_window": "abc"},
"enabled": True})
cap = pool.find_by_model("weird-m")["capabilities"]
assert cap["vision"] is True # 非布尔按缺省
assert cap["context_window"] == 0 # 非整数回落不限
def test_filter_by_capabilities_hard_rules():
entries = [{"capabilities": {"vision": False, "tools": True, "context_window": 8192}},
{"capabilities": {"vision": True, "tools": True, "context_window": 32768}},
{"capabilities": {"vision": True, "tools": False, "context_window": 0}}]
out = filter_by_capabilities(entries, need_vision=True)
assert len(out) == 2
out = filter_by_capabilities(entries, need_vision=True, need_tools=True)
assert len(out) == 1
out = filter_by_capabilities(entries, need_vision=True, need_tools=True,
min_context_tokens=16384)
assert len(out) == 1 and out[0]["capabilities"]["context_window"] == 32768
# context_window=0(不限)不受 min_context 约束
out = filter_by_capabilities(entries, min_context_tokens=999999)
assert len(out) == 1 and out[0]["capabilities"]["context_window"] == 0
def test_request_needs_detection():
body = {"messages": [
{"role": "user", "content": "普通文本问题"},
{"role": "user", "content": [{"type": "text", "text": "看这张图"},
{"type": "image_url",
"image_url": {"url": "data:image/png;base64,x"}}]},
], "tools": [{"type": "function", "function": {"name": "f"}}],
"max_tokens": 3000}
needs = _request_needs(body)
assert needs["vision"] is True and needs["tools"] is True
assert needs["min_context_tokens"] > 3000
plain = _request_needs({"messages": [{"role": "user", "content": "hi"}]})
assert plain["vision"] is False and plain["tools"] is False
assert plain["min_context_tokens"] > 0
def test_entry_meets_defaults_and_limits():
legacy = {"capabilities": {"vision": True, "tools": True, "context_window": 0}}
assert _entry_meets(legacy, {"vision": True, "tools": True,
"min_context_tokens": 999999})
limited = {"capabilities": {"vision": True, "tools": True, "context_window": 4096}}
assert _entry_meets(limited, {"vision": False, "tools": False,
"min_context_tokens": 4096})
assert not _entry_meets(limited, {"vision": False, "tools": False,
"min_context_tokens": 4097})
def test_pool_list_returns_capabilities(tmp_path):
"""list() 打码输出保留 capabilities 字段(前端可见)。"""
pool = _make_pool(tmp_path)
e = next(x for x in pool.list()["entries"] if x["id"] == "vision")
assert e["capabilities"]["vision"] is True
+134
View File
@@ -0,0 +1,134 @@
"""预算四档渐进干预测试(T-X1):budget_mode 黄金边界 + 降档选择器。
黄金用例锁死(整数基点 bp,1% = 100bp):
projected < 8000bp -> normal
8000bp <= projected -> optimize
9500bp <= projected -> cheap
projected > 10000bp -> block(try_hold 硬拒兜底)
"""
import pytest
from gateway.model_pool import PoolStore
from gateway.proxy.billing import BUDGET_CHEAP_BP, BUDGET_OPTIMIZE_BP
from gateway.proxy.ledger import Ledger
from gateway.proxy.ledgerutil import _today
from gateway.proxy.routes import _budget_headers, _downgrade_entry
pytest.importorskip("fastapi")
_TS = 1789874000.0 # 固定时间戳(黄金用例确定性)
def _make_ledger(tmp_path, daily_cap_yuan: float = 10.0) -> Ledger:
led = Ledger.init_db(tmp_path / "budget.sqlite3")
led.upsert_student("张三", "软件2201", balance_yuan=100.0,
daily_cap_yuan=daily_cap_yuan)
return led
def _set_spent(led: Ledger, student_id: int, spent_milli: int,
spent_date: str = "") -> None:
"""直接写当日已花(白盒:budget_mode 的输入口径与 try_hold 一致)。"""
with led._lock, led._connect() as conn:
conn.execute(
"UPDATE students SET spent_today_milli = ?, spent_date = ? WHERE id = ?",
(spent_milli, spent_date, student_id))
def test_threshold_constants_golden():
"""阈值常量黄金锁定:80% / 95%。"""
assert BUDGET_OPTIMIZE_BP == 8000
assert BUDGET_CHEAP_BP == 9500
@pytest.mark.parametrize("spent,est,expected", [
(0, 7999, "normal"), # 7999bp
(0, 8000, "optimize"), # 恰好 8000bp(边界含头)
(0, 9499, "optimize"),
(0, 9500, "cheap"), # 恰好 9500bp(边界含头)
(0, 10000, "cheap"), # 恰好达上限 = cheap(不 block)
(0, 10001, "block"), # 严格超出才 block
(5000, 3000, "optimize"), # 8000bp:spent 与 est 合并计算
(5000, 4500, "cheap"), # 9500bp
(5000, 5001, "block"), # 10001bp
])
def test_budget_mode_golden_boundaries(tmp_path, spent, est, expected):
led = _make_ledger(tmp_path, daily_cap_yuan=10.0) # cap = 10000 毫元
_set_spent(led, 1, spent, spent_date=_today(_TS)) # 记账日 = 判定日(当日口径)
assert led.budget_mode(1, est, ts=_TS) == expected
def test_budget_mode_zero_cap_means_unlimited(tmp_path):
led = _make_ledger(tmp_path, daily_cap_yuan=0.0)
sid = 1
_set_spent(led, sid, 999999, spent_date=_today(_TS))
assert led.budget_mode(sid, 10**9, ts=_TS) == "normal"
def test_budget_mode_spent_resets_by_date(tmp_path):
"""spent_date 非今日 -> 当日已花按 0 计(与 try_hold 口径一致)。"""
led = _make_ledger(tmp_path, daily_cap_yuan=10.0)
_set_spent(led, 1, 9999, spent_date="2000-01-01") # 非今日
assert led.budget_mode(1, 1, ts=1789874000.0) == "normal"
def test_budget_mode_missing_student_normal(tmp_path):
led = Ledger.init_db(tmp_path / "b.sqlite3")
assert led.budget_mode(999, 100, ts=1789874000.0) == "normal"
def _make_pool(tmp_path) -> PoolStore:
import gateway.model_pool as mp
store = PoolStore(path=tmp_path / "pool.json") # 显式隔离,防污染真实池文件
mp.reset_pool()
store.upsert({"id": "p1", "name": "旗舰", "tier": "premium", "backend": "openai",
"base_url": "https://api.example.com", "model": "big-x",
"enabled": True})
store.upsert({"id": "b1", "name": "实惠", "tier": "budget", "backend": "openai",
"base_url": "https://api.example.com/v2", "model": "mid-y",
"enabled": True})
store.upsert({"id": "l1", "name": "本地", "tier": "local",
"backend": "llama_server", "base_url": "http://127.0.0.1:8901",
"model": "qwen-local", "enabled": True})
return store
def test_downgrade_optimize_one_tier(tmp_path):
pool = _make_pool(tmp_path)
premium = pool.find_by_model("big-x")
down = _downgrade_entry(pool, premium, "optimize")
assert down is not None and down["tier"] == "budget"
def test_downgrade_cheap_to_lowest(tmp_path):
pool = _make_pool(tmp_path)
premium = pool.find_by_model("big-x")
down = _downgrade_entry(pool, premium, "cheap")
assert down is not None and down["tier"] == "local"
def test_downgrade_stops_at_local(tmp_path):
"""已在最低档:cheap/optimize 均不再降(杜绝反向升档)。"""
pool = _make_pool(tmp_path)
local = pool.find_by_model("qwen-local")
assert _downgrade_entry(pool, local, "cheap") is None
assert _downgrade_entry(pool, local, "optimize") is None
def test_downgrade_skips_disabled_and_mock(tmp_path):
pool = _make_pool(tmp_path)
pool.upsert({"id": "b2", "name": "停用", "tier": "budget", "backend": "openai",
"base_url": "https://api.example.com/v3", "model": "mid-z",
"enabled": False})
pool.upsert({"id": "mk", "name": "假", "tier": "budget", "backend": "mock",
"model": "mock", "enabled": True})
premium = pool.find_by_model("big-x")
down = _downgrade_entry(pool, premium, "optimize")
assert down is not None and down["id"] == "b1" # 跳过停用与 mock
def test_budget_headers_only_when_abnormal():
assert _budget_headers("normal") == {}
assert _budget_headers("") == {}
assert _budget_headers("optimize") == {"X-Budget-Mode": "optimize"}
assert _budget_headers("cheap") == {"X-Budget-Mode": "cheap"}
+35 -1
View File
@@ -100,7 +100,10 @@ def _auth(key):
return {"Authorization": f"Bearer {key['key']}"} return {"Authorization": f"Bearer {key['key']}"}
BODY = {"model": "deepseek-chat", "messages": [{"role": "user", "content": "问个问题"}]} BODY = {"model": "deepseek-chat",
"messages": [{"role": "user",
"content": "请详细介绍快速排序算法的原理、复杂度与实现要点,"
"并给出 Python 示例代码与适用场景分析。"}]}
def test_chat_non_stream_end_to_end(tmp_path): def test_chat_non_stream_end_to_end(tmp_path):
@@ -288,3 +291,34 @@ def test_openai_protocol_compliance_via_httpx(tmp_path):
pass pass
mp.reset_pool() mp.reset_pool()
reset_auth_state() reset_auth_state()
def test_cache_hit_second_request(tmp_path):
"""T-P6 端到端:首问打上游并写缓存;同问再答 X-Cache: HIT 且上游仅 1 次。"""
from gateway.proxy.routes import reset_semcache_instances
reset_semcache_instances()
tc, ledger, key, calls, (upmod, orig, hclient) = _make_app(tmp_path)
try:
r1 = tc.post("/proxy/v1/chat/completions", json=BODY, headers=_auth(key))
assert r1.status_code == 200
assert r1.headers.get("x-cache") is None # 首问未命中
assert calls["n"] == 1
r2 = tc.post("/proxy/v1/chat/completions", json=BODY, headers=_auth(key))
assert r2.status_code == 200
assert r2.headers.get("x-cache") == "HIT" # 缓存命中
assert calls["n"] == 1 # 上游仍只 1 次
assert r2.json()["choices"][0]["message"]["content"] == "你好,世界"
rid = r2.headers["x-request-id"]
u = ledger.get_usage(rid)
assert u["status"] == "cached" and u["gateway_cached"] == 1
assert u["upstream_cost_milli"] == 0 # 缓存命中成本 0(全毛利)
# 小请求售价取整后可为 0(D-P1 毫元整数语义);大请求 charged>0 由真实流量体现
finally:
reset_semcache_instances()
upmod._client = orig
try:
asyncio_run(hclient.aclose())
except Exception:
pass
mp.reset_pool()
reset_auth_state()
+168
View File
@@ -0,0 +1,168 @@
"""语义缓存测试(T-P6,M2):精确/n-gram 阈值/TTL/LRU/重建/晋升/singleflight/SSE 回放。"""
import asyncio
import itertools
import json
import pytest
from gateway.proxy.semcache import (
SingleFlight,
SemanticCache,
grams,
synth_sse_chunks,
weighted_jaccard,
)
from gateway.proxy.ledger import Ledger
@pytest.fixture()
def cache(tmp_path):
led = Ledger.init_db(tmp_path / "p.sqlite3")
return SemanticCache(led, max_entries=100, sim_threshold=0.92,
promote_frequency=5)
def _put(cache, key, text, answer="答案A", model="m", doc_version=1, ttl_hours=72):
cache.put(key, text, answer, model, doc_version=doc_version, ttl_hours=ttl_hours)
def test_grams_and_weighted_jaccard():
g1 = grams("什么是递归")
assert any(len(g) == 2 for g in g1) and any(len(g) == 3 for g in g1)
assert weighted_jaccard(g1, g1) == 1.0
assert weighted_jaccard(grams("完全不同话题"), g1) == 0.0
def test_exact_hit_and_miss(cache):
key = "default|1|" + "a" * 16
_put(cache, key, "什么是递归", "递归是自调用")
hit = cache.lookup(key, "什么是递归")
assert hit and hit["level"] == "exact" and hit["answer"] == "递归是自调用"
assert cache.lookup(key + "-nope", "完全无关的问题") is None
def test_semantic_hit_above_threshold(cache):
"""同义变体:L2 命中(阈值上)。"""
key = "default|1|b1"
_put(cache, key, "请解释一下什么叫做递归函数", "递归解释")
hit = cache.lookup("default|1|b2", "请解释一下什么叫做递归函数", doc_version=1)
assert hit and hit["level"] == "semantic"
assert cache.hits_semantic == 1
def test_semantic_miss_below_threshold(cache):
"""完全不同语义:未命中(阈值下)。"""
_put(cache, "default|1|c1", "请解释一下什么叫做递归函数", "递归解释")
assert cache.lookup("default|1|c2", "今天股市行情怎么样", doc_version=1) is None
def test_ttl_expiry_fake_clock(cache):
_put(cache, "k", "某个问题文本", "旧答案", ttl_hours=1)
cache._clock = cache._now() + 7200 # 假时钟 +2h
assert cache.lookup("k", "某个问题文本", doc_version=1) is None # 过期不可见
def test_lru_eviction(tmp_path):
led = Ledger.init_db(tmp_path / "p.sqlite3")
cache = SemanticCache(led, max_entries=3)
for i in range(5):
_put(cache, f"k{i}", f"完全不同的问题编号{i}", f"答{i}")
assert len(cache._l1) == 3 # LRU 上限
assert cache.lookup("k0", "完全不同的问题编号0") is None # 最旧被驱逐
assert cache.lookup("k4", "完全不同的问题编号4") is not None
def test_rebuild_from_sqlite(tmp_path):
"""启动时由 semcache 表重建倒排索引。"""
led = Ledger.init_db(tmp_path / "p.sqlite3")
c1 = SemanticCache(led, max_entries=100)
c1.put("k", "解释递归的概念", "持久化答案", "m")
c2 = SemanticCache(led, max_entries=100) # 新实例:重建
hit = c2.lookup("k2", "解释递归的概念", doc_version=1)
assert hit and hit["answer"] == "持久化答案"
def test_promote_after_five_semantic_hits(cache):
"""L2 命中 5 次 -> 晋升 L1(promote 别名键可精确命中)。"""
key = "default|1|p1"
_put(cache, key, "请解释一下什么叫做递归函数呢?", "递归解释(变体)")
promoted = False
for i in range(5):
hit = cache.lookup("default|1|p%d" % (i + 2), "请解释一下什么叫做递归函数呢", doc_version=1)
assert hit and hit["level"] == "semantic"
if any(k.startswith("default|1|promoted:") for k in cache._l1):
promoted = True
assert promoted and cache.hits_semantic == 5
def test_singleflight_merge_and_bypass():
"""两并发同请求:一登记一等待;超限旁路。"""
async def scenario():
sf = SingleFlight()
fut, slot = sf.try_claim("h1")
assert fut is None and slot is not None # 首个登记
fut2, slot2 = sf.try_claim("h1")
assert fut2 is not None and slot2 is None # 第二个等待
sf.release(slot, result="共享答案")
got = await sf.wait(fut2)
assert got == "共享答案"
# 超限旁路
sf2 = SingleFlight()
sf2.MAX = 2
_f, s1 = sf2.try_claim("a")
_f2, s2 = sf2.try_claim("b")
f3, s3 = sf2.try_claim("c")
assert f3 is None and s3 is None # 第三个旁路
asyncio.run(scenario())
def test_synth_sse_chunks_valid():
"""命中回放:合法 SSE 形状(delta 分块 + finish + [DONE])。"""
chunks = synth_sse_chunks("你好世界" * 10, chunk_size=20, model="m", request_id="r1")
text = b"".join(chunks).decode("utf-8")
assert text.count("chat.completion.chunk") >= 2
assert '"finish_reason": "stop"' in text or '"finish_reason":"stop"' in text
assert text.rstrip("\n").endswith("data: [DONE]")
content = ""
done = False
for raw in chunks:
for line in raw.decode("utf-8").splitlines():
if not line.startswith("data:"):
continue
payload = line[5:].strip()
if payload == "[DONE]":
done = True
continue
obj = json.loads(payload)
assert obj["object"] == "chat.completion.chunk"
content += obj["choices"][0]["delta"].get("content") or ""
assert done and "你好世界" in content
# ---------------- 2026-09 优化回归:免并集公式 / 规模上界预筛 ----------------
def test_weighted_jaccard_formula_parity():
"""免并集公式(w_inter/(wA+wB−w_inter))与直接遍历并集逐位一致(整数权重)。"""
def direct(ga: set, gb: set) -> float:
inter = ga & gb
if not inter:
return 0.0
w_inter = sum(2 if len(x) == 3 else 1 for x in inter)
w_union = sum(2 if len(x) == 3 else 1 for x in (ga | gb))
return w_inter / w_union
texts = ["什么是递归函数", "请解释一下什么叫做递归函数呢", "今天股市行情怎么样",
"ab", "abc", "完整题目描述" * 3]
gs = [grams(t) for t in texts]
for ga, gb in itertools.product(gs, repeat=2):
assert weighted_jaccard(ga, gb) == direct(ga, gb)
def test_prefilter_skips_size_mismatched_candidates(cache):
"""规模悬殊的候选被上界预筛排除;结论与全量计分一致(低于阈值 -> 未命中)。"""
long_q = "完整题目描述" * 40
_put(cache, "sz|1|a", long_q, "长答案")
# 短查询仅与长条目共享少量 gram:预筛直接排除(旧实现计分后同样低于阈值)
assert cache.lookup("sz|1|b", "完整题目", doc_version=1) is None
assert cache.hits_semantic == 0
+130
View File
@@ -0,0 +1,130 @@
"""上游有序降级链测试(T-X2):链构造排序 + 首 token 前 failover 三元组上报。"""
import asyncio
import json
import httpx
import pytest
from gateway.model_pool import PoolStore
from gateway.proxy.routes import _fallback_chain
from gateway.proxy.upstream import UpstreamError, failover_stats, reset_failover_stats
pytest.importorskip("fastapi")
def _make_pool(tmp_path) -> PoolStore:
store = PoolStore(path=tmp_path / "pool.json")
store.upsert({"id": "p-main", "name": "主", "tier": "premium", "backend": "openai",
"base_url": "http://bad", "model": "big-x", "price_in": 1.0,
"price_out": 2.0, "enabled": True})
store.upsert({"id": "p-cheap", "name": "便宜云", "tier": "budget", "backend": "openai",
"base_url": "http://good", "model": "mid-y", "price_in": 0.1,
"price_out": 0.2, "enabled": True})
store.upsert({"id": "p-local", "name": "本地", "tier": "local",
"backend": "llama_server", "base_url": "http://local:8901",
"model": "qwen-local", "price_in": 0.0, "price_out": 0.0,
"enabled": True})
store.upsert({"id": "p-off", "name": "停用", "tier": "local",
"backend": "openai", "base_url": "http://off", "model": "off-z",
"enabled": False})
store.upsert({"id": "p-mock", "name": "假", "tier": "local", "backend": "mock",
"model": "mock", "enabled": True})
return store
def test_fallback_chain_order_and_exclusions(tmp_path):
"""链序:档位升序 -> 单价和升序;排除 mock/停用/同模型;截断到 max_total。"""
pool = _make_pool(tmp_path)
main = pool.find_by_model("big-x")
chain = _fallback_chain(pool, main)
assert [e["model"] for e in chain] == ["qwen-local", "mid-y"] # local(0) < budget(1)
# 预算降档后的条目作主条目:不回排同档更贵者之外的高价条目在前
short = _fallback_chain(pool, main, max_total=2)
assert [e["model"] for e in short] == ["qwen-local"]
def test_fallback_chain_excludes_same_model(tmp_path):
"""同模型不同条目不进链(避免重复打同一上游)。"""
pool = _make_pool(tmp_path)
pool.upsert({"id": "p-dup", "name": "重", "tier": "budget", "backend": "openai",
"base_url": "http://dup", "model": "mid-y", "enabled": True})
mid = pool.find_by_model("mid-y")
assert all(e["model"] != "mid-y" for e in _fallback_chain(pool, mid))
def test_stream_failover_reports_triplet():
"""主上游首 token 前 500 -> 切换候选成功;sink 与统计如实上报三元组。"""
reset_failover_stats()
calls: list = []
def handler(request: httpx.Request) -> httpx.Response:
calls.append(str(request.url))
if request.url.host == "bad":
return httpx.Response(500, text="boom")
body = ("\n\n".join([
'data: {"choices":[{"delta":{"content":"OK"}}]}',
"data: [DONE]",
]) + "\n\n").encode("utf-8")
return httpx.Response(200, content=body)
import gateway.proxy.upstream as up
client = httpx.AsyncClient(transport=httpx.MockTransport(handler))
orig = up._client
up._client = client
primary = {"base_url": "http://bad", "model": "big-x", "provider": "openai"}
fallback = {"base_url": "http://good", "model": "mid-y", "provider": "openai"}
async def run():
sink: dict = {}
chunks = []
async for b in up.stream({"model": "big-x", "messages": []}, primary, sink,
[fallback]):
chunks.append(b)
return sink, chunks
try:
sink, chunks = asyncio.run(run())
finally:
up._client = orig
assert b"OK" in b"".join(chunks)
fb = sink["upstream_fallback"]
assert fb["used"] is True
assert fb["original"] == "big-x"
assert fb["used_model"] == "mid-y"
assert "500" in fb["reason"]
stats = failover_stats()
assert stats["total"] == 1
assert stats["recent"][0]["original"] == "big-x"
reset_failover_stats()
assert failover_stats()["total"] == 0
def test_stream_all_fail_raises_and_counts():
"""全部候选失败 -> UpstreamError;每次首 token 前失败均计入统计。"""
reset_failover_stats()
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(503, text="down")
import gateway.proxy.upstream as up
client = httpx.AsyncClient(transport=httpx.MockTransport(handler))
orig = up._client
up._client = client
primary = {"base_url": "http://bad", "model": "big-x", "provider": "openai"}
fallback = {"base_url": "http://good", "model": "mid-y", "provider": "openai"}
async def run():
sink: dict = {}
async for _b in up.stream({"model": "big-x", "messages": []}, primary, sink,
[fallback]):
pass
try:
with pytest.raises(UpstreamError):
asyncio.run(run())
finally:
up._client = orig
assert failover_stats()["total"] == 2 # 两个候选各记一次
reset_failover_stats()
+16 -6
View File
@@ -57,14 +57,24 @@ def test_should_enqueue_force_safety():
force_tags=["safety"]) is False force_tags=["safety"]) is False
class _DetRng:
"""极简确定性伪随机(LCG):抽样测试用,避免依赖 random 模块的全局状态。"""
def __init__(self, seed: int):
self._s = seed & 0x7FFFFFFF or 1
def random(self) -> float:
self._s = (1103515245 * self._s + 12345) & 0x7FFFFFFF
return self._s / 0x7FFFFFFF
def test_should_enqueue_sample_rate(): def test_should_enqueue_sample_rate():
import random # 确定性伪随机下按抽样率应命中/不命中可控
# 固定随机种子下按 10% 抽样应命中/不命中可控 hit = sum(ReviewQueue.should_enqueue(["code"], sample_rate=0.0, force_tags=[],
rng = random.Random(42) rng=_DetRng(42)) for _ in range(1000))
hit = sum(ReviewQueue.should_enqueue(["code"], sample_rate=0.0, force_tags=[], rng=rng) for _ in range(1000))
assert hit == 0 # sample_rate=0 -> 永不抽样 assert hit == 0 # sample_rate=0 -> 永不抽样
rng = random.Random(1) hit = sum(ReviewQueue.should_enqueue(["code"], sample_rate=1.0, force_tags=[],
hit = sum(ReviewQueue.should_enqueue(["code"], sample_rate=1.0, force_tags=[], rng=rng) for _ in range(10)) rng=_DetRng(1)) for _ in range(10))
assert hit == 10 # sample_rate=1 -> 全抽样 assert hit == 10 # sample_rate=1 -> 全抽样
+62
View File
@@ -0,0 +1,62 @@
"""五维评分选模与拒绝理由留痕(T-X13,采纳 ai-model-router scoring 设计)。"""
import gateway.proxy.routes as R
from gateway.proxy.routes import (_entry_reject_reason, _rank_candidates,
_record_route_score_event,
reset_route_score_events)
def _entry(model, tier="budget", price_in=1.0, price_out=2.0,
vision=True, tools=True, ctx=0):
return {"id": model, "model": model, "enabled": True,
"backend": "openai", "base_url": "http://127.0.0.1:9/v1",
"tier": tier, "price_in": price_in, "price_out": price_out,
"capabilities": {"vision": vision, "tools": tools,
"context_window": ctx}}
_NEEDS = {"vision": True, "tools": False, "min_context_tokens": 1000}
def test_reject_reasons_are_human_readable():
"""vision/tools/上下文三类不满足均有对应人话理由;满足则 None。"""
assert "vision" in (_entry_reject_reason(_entry("a", vision=False), _NEEDS) or "")
need_tools = {"vision": False, "tools": True, "min_context_tokens": 1000}
assert "tools" in (_entry_reject_reason(_entry("a", tools=False), need_tools) or "")
small = _entry("a", ctx=512)
assert "512" in (_entry_reject_reason(small, _NEEDS) or "")
assert _entry_reject_reason(_entry("a", ctx=8192), _NEEDS) is None
def test_rank_prefers_capable_and_cheap():
"""不满足需求者 capability=0 沉底;便宜者靠成本效率维胜出。"""
good_cheap = _entry("cheap", tier="budget", price_in=0.1, price_out=0.2, ctx=8192)
good_pricey = _entry("pricey", tier="premium", price_in=8.0, price_out=8.0, ctx=8192)
ranked = _rank_candidates([good_pricey, good_cheap], _NEEDS)
assert ranked[0]["model"] == "cheap"
assert ranked[0]["dims"]["capability"] > 0
assert ranked[0]["dims"]["cost_efficiency"] > ranked[1]["dims"]["cost_efficiency"]
# premium 的质量维更高(启发式)
assert ranked[1]["dims"]["quality"] > ranked[0]["dims"]["quality"]
def test_rank_tie_break_deterministic():
"""同配置同分按模型名字典序。"""
a, b = _entry("zeta"), _entry("alpha")
ranked = _rank_candidates([a, b], _NEEDS)
assert [s["model"] for s in ranked] == ["alpha", "zeta"]
def test_score_event_records_rejections():
"""评分留痕:入选排序 + 被淘汰候选的拒绝理由;reset 清空。"""
reset_route_score_events()
ranked = _rank_candidates([_entry("good", ctx=8192)], _NEEDS)
rejected = [{"model": "bad", "reason": "模型不支持视觉输入(vision)"}]
_record_route_score_event("px-test", ranked, rejected)
_record_route_score_event("px-test-2", ranked, rejected)
assert len(R._route_score_events) == 2
ev = R._route_score_events[-1]
assert ev["request_id"] == "px-test-2"
assert ev["ranked"][0]["model"] == "good"
assert ev["rejected"][0]["reason"].startswith("模型不支持")
reset_route_score_events()
assert len(R._route_score_events) == 0
+87
View File
@@ -0,0 +1,87 @@
"""语义缓存路由签名分桶(T-X10,采纳 cortiq 路由签名设计)。"""
from gateway.proxy.config import build_proxy_config
from gateway.proxy.normalizer import canonical_hash
from gateway.proxy.routes import _route_sig
_BODY = {"model": "m1", "messages": [{"role": "user", "content": "什么是递归"}]}
def test_canonical_hash_legacy_format_without_sig():
"""不带签名保持旧三段格式(旧调用/旧条目兼容)。"""
h = canonical_hash("default", 1, _BODY)
assert h.count("|") == 2
h2 = canonical_hash("default", 1, _BODY, "")
assert h == h2
def test_canonical_hash_with_sig_divides_keyspace():
"""签名不同 -> 键不同:vision 请求与纯文本请求不再共用缓存条目。"""
plain = canonical_hash("default", 1, _BODY)
vision = canonical_hash("default", 1, _BODY, "v1t0")
tools = canonical_hash("default", 1, _BODY, "v0t1")
assert len({plain, vision, tools}) == 3
def test_route_sig_scopes():
"""三种 scope 的签名形态:capabilities 默认 / model 精确 / none 旧格式。"""
cfg = build_proxy_config({"proxy": {"enabled": True}})
needs = {"vision": True, "tools": False, "min_context_tokens": 10}
assert _route_sig(needs, "m1", cfg) == "v1t0"
cfg_model = build_proxy_config({"proxy": {"semcache": {"route_sig_scope": "model"}}})
assert _route_sig(needs, "m1", cfg_model) == "m:m1"
cfg_none = build_proxy_config({"proxy": {"semcache": {"route_sig_scope": "none"}}})
assert _route_sig(needs, "m1", cfg_none) == ""
# 非法 scope 回落 capabilities
cfg_bad = build_proxy_config({"proxy": {"semcache": {"route_sig_scope": "bogus"}}})
assert _route_sig(needs, "m1", cfg_bad) == "v1t0"
def test_sig_isolation_end_to_end_via_semcache(tmp_path):
"""经 SemanticCache 验证:签名分区 L1 精确层与 L2 语义扫描两层。"""
from gateway.proxy.ledger import Ledger
from gateway.proxy.semcache import SemanticCache
led = Ledger.init_db(tmp_path / "l.sqlite3")
cache = SemanticCache(led, max_entries=100, sim_threshold=0.5,
promote_frequency=5)
k_text = canonical_hash("default", 1, _BODY, "v0t0")
k_vision = canonical_hash("default", 1, _BODY, "v1t0")
cache.put(k_text, "什么是递归", "纯文本答案", "m1", doc_version=1,
route_sig="v0t0")
# 相同问题、vision 签名:L1 键不同必不中;L2 语义扫描按签名分区也不中
assert cache.lookup(k_vision, "什么是递归", doc_version=1,
route_sig="v1t0") is None
# 同签名:L1 精确命中
hit = cache.lookup(k_text, "什么是递归", doc_version=1, route_sig="v0t0")
assert hit is not None and hit["level"] == "exact"
def test_sig_partition_survives_rebuild(tmp_path):
"""重启重建(semcache 表无签名列):签名从缓存键解析恢复,分区仍有效。"""
from gateway.proxy.ledger import Ledger
from gateway.proxy.semcache import SemanticCache
led = Ledger.init_db(tmp_path / "l.sqlite3")
c1 = SemanticCache(led, max_entries=100, sim_threshold=0.5, promote_frequency=5)
k = canonical_hash("default", 1, _BODY, "v1t0")
c1.put(k, "什么是递归", "多模态答案", "m1", doc_version=1, route_sig="v1t0")
c2 = SemanticCache(led, max_entries=100, sim_threshold=0.5, promote_frequency=5)
assert c2.lookup(k, "什么是递归", doc_version=1, route_sig="v1t0") is not None
assert c2.lookup(canonical_hash("default", 1, _BODY, "v0t0"),
"什么是递归", doc_version=1, route_sig="v0t0") is None
def test_legacy_calls_without_sig_unchanged(tmp_path):
"""不传签名(route_sig=None)的旧调用零过滤:既有测试行为不变。"""
from gateway.proxy.ledger import Ledger
from gateway.proxy.semcache import SemanticCache
led = Ledger.init_db(tmp_path / "l.sqlite3")
cache = SemanticCache(led, max_entries=100, sim_threshold=0.5, promote_frequency=5)
cache.put("k1", "请解释一下递归函数的概念", "答案", "m1", doc_version=1)
assert cache.lookup("k1", "请解释一下递归函数的概念", doc_version=1) is not None
# 语义路径同样不受影响(近似文本跨键命中)
assert cache.lookup("k2", "请解释一下递归函数的概念呢", doc_version=1) is not None
+157
View File
@@ -0,0 +1,157 @@
"""决策缓存测试(T-X3):LRU+TTL 单元契约 + Grader live 模式接线(embed 去重)。"""
import asyncio
import pytest
from gateway.sense.config import build_sense_config
from gateway.sense.decision_cache import DecisionCache
from gateway.sense.grader import Grader
from gateway.sense.store import SenseStore
# ---------------- DecisionCache 单元 ----------------
class _Clock:
def __init__(self):
self.t = 100.0
def __call__(self):
return self.t
def test_key_scope_and_text_sensitive():
assert DecisionCache.key("proxy", "abc") == DecisionCache.key("proxy", "abc")
assert DecisionCache.key("proxy", "abc") != DecisionCache.key("pipeline", "abc")
assert DecisionCache.key("proxy", "abc") != DecisionCache.key("proxy", "abd")
def test_put_get_hit_miss_counters():
c = DecisionCache()
assert c.get("s", "q") is None
c.put("s", "q", {"tier": "T1"})
assert c.get("s", "q") == {"tier": "T1"}
st = c.stats()
assert st["hits"] == 1 and st["misses"] == 1 and st["entries"] == 1
def test_ttl_expiry_with_fake_clock():
clock = _Clock()
c = DecisionCache(ttl_s=60.0, now=clock)
c.put("s", "q", {"v": 1})
assert c.get("s", "q") == {"v": 1}
clock.t += 61.0 # 超时
assert c.get("s", "q") is None
assert c.stats()["entries"] == 0 # 过期条目被清除
def test_lru_eviction_at_maxsize():
c = DecisionCache(maxsize=2)
c.put("s", "a", 1)
c.put("s", "b", 2)
c.get("s", "a") # a 变为最近使用
c.put("s", "c", 3) # 淘汰 b
assert c.get("s", "a") == 1
assert c.get("s", "b") is None
assert c.get("s", "c") == 3
def test_clear_keeps_counters_but_drops_entries():
c = DecisionCache()
c.put("s", "q", 1)
c.get("s", "q")
c.clear()
assert c.get("s", "q") is None
assert c.stats()["hits"] == 1 # 计数保留(审计口径不回退)
# ---------------- Grader live 模式接线 ----------------
class _CountingObserver:
def __init__(self):
self.logs = []
def log(self, obs):
self.logs.append(obs)
class _FakeHead:
version = "test-head"
def predict(self, vec):
return {"t1": 0.95, "t2": 0.03, "t3": 0.02}
def _make_grader(tmp_path, mode, counter, observer=None):
"""构造测试 Grader:embed 打点计数 + FakeHead 注入。"""
import gateway.sense.grader as gr
cfg = build_sense_config({"sense": {
"enabled": True, "mode": mode,
"db_path": str(tmp_path / "s.sqlite3"),
"models_dir": str(tmp_path / "models")}})
store = SenseStore.init_db(cfg.db_path)
g = Grader(cfg, store, observer or _CountingObserver())
g._load_head = lambda: _FakeHead()
async def fake_embed(text, c):
counter["n"] += 1
return [1, 2, 3]
orig = gr.embed
gr.embed = fake_embed
return g, orig
def test_grader_live_mode_dedupes_embed(tmp_path):
"""live 模式:同文本第二次 decide 命中缓存,embed 只跑一次;观察仍写两条。"""
import gateway.sense.grader as gr
counter = {"n": 0}
observer = _CountingObserver()
g, orig = _make_grader(tmp_path, "live", counter, observer)
try:
async def run():
d1 = await g.decide("什么是递归", "proxy")
d2 = await g.decide("什么是递归", "proxy")
d3 = await g.decide("换一个问题", "proxy")
return d1, d2, d3
d1, d2, d3 = asyncio.run(run())
finally:
gr.embed = orig
assert d1.tier == "T1" and d2.tier == "T1" and d3.tier == "T1"
assert counter["n"] == 2 # 同文本去重:2 个不同文本各 1 次
assert d2.head_version == d1.head_version
assert len(observer.logs) == 3 # 观察不因缓存命中而跳过
def test_grader_collect_mode_no_cache(tmp_path):
"""collect/shadow:校准数据必须全量产出,不走缓存。"""
import gateway.sense.grader as gr
counter = {"n": 0}
g, orig = _make_grader(tmp_path, "collect", counter)
try:
async def run():
await g.decide("同一个问题", "proxy")
await g.decide("同一个问题", "proxy")
asyncio.run(run())
finally:
gr.embed = orig
assert counter["n"] == 2
def test_grader_invalidate_clears_cache(tmp_path):
"""invalidate(工件切换)后缓存清空:同文本重新 embed。"""
import gateway.sense.grader as gr
counter = {"n": 0}
g, orig = _make_grader(tmp_path, "live", counter)
try:
async def run():
await g.decide("什么是递归", "proxy")
g.invalidate()
await g.decide("什么是递归", "proxy")
asyncio.run(run())
finally:
gr.embed = orig
assert counter["n"] == 2
+200
View File
@@ -0,0 +1,200 @@
"""决策留痕测试(T-X4):store 三元结构落库 + grader 留痕 + admin 查询端点。"""
import asyncio
import pytest
pytest.importorskip("fastapi")
from fastapi import FastAPI
from fastapi.testclient import TestClient
import gateway.model_pool as mp
import gateway.sense.grader as gr
from gateway.sense.config import build_sense_config
from gateway.sense.grader import Grader
from gateway.sense.store import SenseStore
from gateway.proxy.config import build_proxy_config
from gateway.proxy.routes import build_proxy_router
class _CountingObserver:
def __init__(self):
self.logs = []
def log(self, obs):
self.logs.append(obs)
class _FakeHead:
version = "test-head"
def predict(self, vec):
return {"t1": 0.95, "t2": 0.03, "t3": 0.03}
def _make_grader(tmp_path, mode="live", embed_raises=False):
cfg = build_sense_config({"sense": {
"enabled": True, "mode": mode,
"db_path": str(tmp_path / "s.sqlite3"),
"models_dir": str(tmp_path / "models")}})
store = SenseStore.init_db(cfg.db_path)
g = Grader(cfg, store, _CountingObserver())
g._load_head = lambda: _FakeHead()
async def fake_embed(text, c):
if embed_raises:
raise __import__("gateway.sense.errors",
fromlist=["EmbedderDown"]).EmbedderDown("down")
return [1, 2, 3]
orig = gr.embed
gr.embed = fake_embed
return g, store, orig
def test_store_decision_roundtrip(tmp_path):
store = SenseStore.init_db(tmp_path / "s.sqlite3")
store.insert_decision({
"ts": 1789874000, "request_id": "rt1", "consumer": "proxy",
"decided_tier": "T1", "executed_tier": "T1", "mode": "live",
"fallback": False, "policy_version": "head:test|th:conservative",
"q_hash": "abc123",
"reasons": '["t1_ok", "p1>=threshold"]',
"candidate_scores": '{"t1": 0.95, "t2": 0.03, "t3": 0.03}',
"rejected": '[{"tier": "T3", "reason": "p3<threshold 且无 repo_signals"}]',
})
rows = store.list_decisions(limit=10)
assert len(rows) == 1
r = rows[0]
assert r["decided_tier"] == "T1" and r["mode"] == "live"
assert r["reasons"][0] == "t1_ok"
assert r["candidate_scores"]["t1"] == 0.95
assert r["rejected"][0]["tier"] == "T3"
def test_grader_records_decision_triplet(tmp_path):
"""正常决策:reasons/candidate_scores/rejected 全量落库。"""
g, store, orig = _make_grader(tmp_path)
try:
asyncio.run(g.decide("什么是递归", "proxy"))
finally:
gr.embed = orig
rows = store.list_decisions()
assert len(rows) == 1
r = rows[0]
assert r["decided_tier"] == "T1"
assert "t1_ok" in r["reasons"] and "p1>=threshold" in r["reasons"]
assert r["rejected"][0]["tier"] == "T3"
assert r["candidate_scores"]["t1"] == 0.95
assert r["q_hash"] and len(r["q_hash"]) == 32
def test_grader_records_fallback_reasons(tmp_path):
"""embedder 故障:fallback 规则门决策同样留痕并标注 fallback。"""
g, store, orig = _make_grader(tmp_path, embed_raises=True)
try:
d = asyncio.run(g.decide("什么是递归", "proxy"))
finally:
gr.embed = orig
assert d.fallback is True
rows = store.list_decisions()
assert len(rows) == 1
r = rows[0]
assert r["fallback"] == 1
assert "fallback:rule_gate" in r["reasons"]
assert {x["reason"] for x in r["rejected"]} == {"fallback"}
def test_admin_sense_decisions_endpoint(tmp_path, monkeypatch):
"""管理面查询端点:sense 未注入/未启用 -> enabled False;不报错。"""
import gateway.model_pool as mp
mp.reset_pool()
monkeypatch.setattr(mp, "_store",
__import__("gateway.model_pool",
fromlist=["PoolStore"]).PoolStore(
path=tmp_path / "pool.json"))
cfg = build_proxy_config({"proxy": {"enabled": True,
"db_path": str(tmp_path / "p.sqlite3")}})
app = FastAPI()
app.include_router(build_proxy_router(cfg, mp.get_pool()))
client = TestClient(app)
r = client.get("/proxy/admin/sense-decisions")
assert r.status_code == 200
body = r.json()
assert body["enabled"] is False and body["decisions"] == []
def test_proxy_routes_sense_live_regression(tmp_path, monkeypatch):
"""T-X4 回归:settings_provider 注入后,sense live 分流真正生效
(修复前 NameError 被 except 吞掉,x-campus-tier 永远缺失)。"""
import httpx
import gateway.proxy.upstream as up
from gateway.proxy.auth import issue_key
from gateway.proxy.ledger import Ledger
sense_dir = tmp_path / "sense"
cfg = build_proxy_config({"proxy": {
"enabled": True, "db_path": str(tmp_path / "p.sqlite3"),
"pricing": {"m1": {"in_miss": 1.0, "out": 2.0}}}})
# 签发学生 key(代理面鉴权链:Bearer 形态 -> 哈希 -> 日限额 -> rpm)
led = Ledger.init_db(tmp_path / "p.sqlite3")
sid = led.upsert_student("测试生", "软件2201", balance_yuan=10.0,
daily_cap_yuan=10.0)
issued = issue_key(led, sid, rpm_cap=100, day_cap_req=1000)
auth_header = {"Authorization": f"Bearer {issued['key']}"}
import gateway.model_pool as mp
mp.reset_pool()
pool = mp.PoolStore(path=tmp_path / "pool.json")
pool.upsert({"id": "t1e", "name": "本地", "tier": "local",
"backend": "openai", "base_url": "http://mockup",
"model": "local-m", "enabled": True})
class _Settings:
def to_dict(self):
return {"sense": {
"enabled": True, "mode": "live",
"db_path": str(sense_dir / "s.sqlite3"),
"models_dir": str(sense_dir / "models"),
"consumers": {"proxy": {"t1": "local", "t2": "budget",
"t3": "premium"}}}}
app = FastAPI()
app.include_router(build_proxy_router(cfg, pool, settings_provider=_Settings))
def handler(request: httpx.Request) -> httpx.Response:
body = ("\n\n".join([
'data: {"choices":[{"delta":{"content":"hi"}}]}',
"data: [DONE]",
]) + "\n\n").encode("utf-8")
return httpx.Response(200, content=body)
client = httpx.AsyncClient(transport=httpx.MockTransport(handler))
orig_client = up._client
up._client = client
async def fake_embed(text, c):
return [1, 2, 3]
orig_embed = gr.embed
gr.embed = fake_embed
try:
c = TestClient(app)
r = c.post("/proxy/v1/chat/completions",
json={"model": "local-m",
"messages": [{"role": "user", "content": "什么是递归"}]},
headers=auth_header)
assert r.status_code == 200
# 短文本 + t1_hard_ok + p1>=0.9 -> T1 生效并替换为 local 档模型
assert r.headers.get("x-campus-tier") == "T1"
assert r.json()["model"] == "local-m"
# 决策留痕可经管理面查询
d = c.get("/proxy/admin/sense-decisions")
assert d.status_code == 200
decisions = d.json()["decisions"]
assert len(decisions) == 1 and decisions[0]["decided_tier"] == "T1"
finally:
up._client = orig_client
gr.embed = orig_embed
mp.reset_pool()
+178
View File
@@ -0,0 +1,178 @@
"""晋升表测试(T-X5):candidate→promoted→demoted→恢复状态机 + 一票否决 +
grader 晋升应用(T2→T1)与 T3/闸门互斥。"""
import asyncio
import pytest
from gateway.sense.config import build_sense_config
from gateway.sense.grader import Grader
from gateway.sense.promotion import PromotionTable, promotion_label
from gateway.sense.store import SenseStore
class _Clock:
def __init__(self, t=1789874000.0):
self.t = t
def __call__(self):
return self.t
def _table(tmp_path, **kw):
store = SenseStore.init_db(tmp_path / "s.sqlite3")
clock = _Clock()
table = PromotionTable(store, now=clock, **kw)
return store, clock, table
def test_promotion_requires_min_and_soak(tmp_path):
"""n_min 未满不晋升; soak 期未满不晋升;两者满足才 promoted。"""
store, clock, table = _table(tmp_path, n_min=5, promote_lb=0.9,
soak_days=3.0)
label = "proxy:general"
for _ in range(5):
table.observe(label, ok=True, ts=clock.t) # 通过率 1.0 但浸泡期未满
assert not table.is_promoted(label)
clock.t += 4 * 86400.0 # 浸泡期满足
table.observe(label, ok=True, ts=clock.t)
assert table.is_promoted(label)
def test_promotion_demotes_on_degradation(tmp_path):
"""promoted 后通过率跌破 demote_lb -> 自动降级。"""
store, clock, table = _table(tmp_path, n_min=5, promote_lb=0.9,
demote_lb=0.85, soak_days=0.0)
label = "proxy:general"
for _ in range(6):
table.observe(label, ok=True, ts=clock.t)
assert table.is_promoted(label)
for _ in range(6):
clock.t += 60
table.observe(label, ok=False, ts=clock.t) # 通过率跌至 6/12 = 0.5
assert not table.is_promoted(label)
assert table.snapshot(label)["state"] == "demoted"
def test_promotion_recovery_after_demotion(tmp_path):
"""demoted 后凭数据恢复:再次满足晋升门 -> promoted。"""
store, clock, table = _table(tmp_path, n_min=4, promote_lb=0.9,
demote_lb=0.85, soak_days=0.0)
label = "proxy:general"
for _ in range(4):
table.observe(label, ok=True, ts=clock.t)
assert table.is_promoted(label)
for _ in range(4):
clock.t += 60
table.observe(label, ok=False, ts=clock.t) # 跌破
assert table.snapshot(label)["state"] == "demoted"
for _ in range(20):
clock.t += 60
table.observe(label, ok=True, ts=clock.t) # 20/24 ≈ 0.83... 继续
# 24 条中 20 ok = 0.833 < 0.9,再补 ok 至 >0.9
for _ in range(6):
clock.t += 60
table.observe(label, ok=True, ts=clock.t)
# 30 条 26 ok ≈ 0.867,仍 <0.9;再补
for _ in range(6):
clock.t += 60
table.observe(label, ok=True, ts=clock.t)
# 36 条 32 ok ≈ 0.889;最后一条后 >= 0.9
table.observe(label, ok=True, ts=clock.t + 60) # 37 条 33 ok ≈ 0.892
for _ in range(4):
table.observe(label, ok=True, ts=clock.t + 120)
# 41 条 37 ok ≈ 0.902 >= 0.9
assert table.snapshot(label)["state"] == "promoted"
def test_high_tier_veto(tmp_path):
"""T3 观察:不计入通过率;已晋升立即降级。"""
store, clock, table = _table(tmp_path, n_min=3, promote_lb=0.9, soak_days=0.0)
label = "proxy:general"
for _ in range(3):
table.observe(label, ok=True, ts=clock.t)
assert table.is_promoted(label)
before = table.snapshot(label)["n_total"]
table.observe(label, ok=True, tier="T3", ts=clock.t + 10)
snap = table.snapshot(label)
assert snap["n_total"] == before # T3 不进统计
assert snap["state"] == "demoted" # 一票否决
def test_promotion_label_format():
assert promotion_label("proxy", "") == "proxy:general"
assert promotion_label("proxy", "code") == "proxy:code"
# ---------------- grader 晋升应用 ----------------
class _FakeHeadMid:
version = "test-head"
def predict(self, vec):
return {"t1": 0.50, "t2": 0.47, "t3": 0.03} # 低于阈值 -> default T2
class _NoopObserver:
def log(self, obs):
pass
def _make(tmp_path, table):
cfg = build_sense_config({"sense": {
"enabled": True, "mode": "live",
"db_path": str(tmp_path / "s.sqlite3"),
"models_dir": str(tmp_path / "models")}})
store = SenseStore.init_db(cfg.db_path)
g = Grader(cfg, store, _NoopObserver(), promotion=table)
g._load_head = lambda: _FakeHeadMid()
async def fake_embed(text, c):
return [1, 2, 3]
import gateway.sense.grader as gr
orig = gr.embed
gr.embed = fake_embed
return g, orig
def test_grader_applies_promotion_t2_to_t1(tmp_path):
"""已晋升标签:T2 决策升级 T1 且 hard_gates 标注 promotion_applied。"""
store = SenseStore.init_db(tmp_path / "s.sqlite3")
clock = _Clock()
table = PromotionTable(store, now=clock, n_min=2, promote_lb=0.9, soak_days=0.0)
label = promotion_label("proxy", "general")
table.observe(label, ok=True, ts=clock.t)
table.observe(label, ok=True, ts=clock.t)
g, orig = _make(tmp_path, table)
import gateway.sense.grader as gr
try:
d = asyncio.run(g.decide("什么是递归", "proxy"))
finally:
gr.embed = orig
assert d.tier == "T1"
assert d.hard_gates["promotion_applied"] is True
def test_grader_skips_promotion_when_not_promoted(tmp_path):
"""未晋升:同样的 T2 决策保持 T2。"""
store = SenseStore.init_db(tmp_path / "s2.sqlite3")
table = PromotionTable(store, now=_Clock(), n_min=100, soak_days=0.0)
g, orig = _make(tmp_path, table)
import gateway.sense.grader as gr
try:
d = asyncio.run(g.decide("什么是递归", "proxy"))
finally:
gr.embed = orig
assert d.tier == "T2"
assert d.hard_gates["promotion_applied"] is False
def test_grader_promotion_not_applied_without_table(tmp_path):
"""未接晋升表(向后兼容):行为与既有决策一致。"""
g, orig = _make(tmp_path, None)
import gateway.sense.grader as gr
try:
d = asyncio.run(g.decide("什么是递归", "proxy"))
finally:
gr.embed = orig
assert d.tier == "T2"
+128
View File
@@ -0,0 +1,128 @@
"""shadow 旁路评审单元测试(T-X11,采纳 cortiq shadow bypass 设计)。
全部走 monkeypatch 假上游,不依赖真实模型/API key。
"""
import asyncio
import json
import pytest
import gateway.proxy.routes as R
from gateway.proxy.routes import (_grade_answer, _pick_review_entry,
_shadow_review_settings,
reset_shadow_review_stats)
from gateway.sense.promotion import PromotionTable, promotion_label
@pytest.fixture(autouse=True)
def _reset():
reset_shadow_review_stats()
yield
reset_shadow_review_stats()
def _fake_upstream(responses):
"""构造假 upstream_stream:按调用顺序吐预定 verdict 文本(SSE 形态)。"""
calls = {"n": 0}
def _stream(body, entry, sink, chain):
async def gen():
idx = min(calls["n"], len(responses) - 1)
calls["n"] += 1
verdict = responses[idx]
chunk = {"choices": [{"delta": {"content": verdict}}]}
yield (f"data: {json.dumps(chunk)}\n\n").encode("utf-8")
yield b"data: [DONE]\n\n"
return gen()
return _stream, calls
def _entry(tier="premium", model="cloud-x"):
return {"id": "e1", "enabled": True, "backend": "openai",
"base_url": "http://127.0.0.1:9/v1", "model": model, "tier": tier}
def test_settings_default_disabled_and_clamped():
"""默认关闭;采样率/超时非法值被夹取。"""
s = _shadow_review_settings({})
assert s["enabled"] is False and s["sample_rate"] == 0.1
assert _shadow_review_settings({"sense": {"shadow_review": {
"enabled": True, "sample_rate": 5, "timeout_s": -3}}})["sample_rate"] == 1.0
assert _shadow_review_settings({"sense": {"shadow_review": {
"enabled": True, "timeout_s": -3}}})["timeout_s"] == 1.0
def test_pick_review_entry_prefers_tier():
"""评审条目优先指定档位,缺失时回退任意启用条目。"""
pool_entries = [_entry("budget", "b1"), _entry("premium", "p1")]
class _P:
@staticmethod
def usable_entries():
return list(pool_entries)
assert _pick_review_entry(_P, "premium")["model"] == "p1"
assert _pick_review_entry(_P, "local")["model"] in ("b1", "p1")
def test_grade_answer_parses_pass_fail_and_error(monkeypatch):
"""PASS/FAIL 解析;上游异常回 ERROR 而非抛出。"""
fake, calls = _fake_upstream(["PASS", "FAIL"])
monkeypatch.setattr(R, "upstream_stream", fake)
e = _entry()
assert asyncio.run(_grade_answer(e, "问", "答", "cloud-x", 5)) == "PASS"
assert asyncio.run(_grade_answer(e, "问", "答", "cloud-x", 5)) == "FAIL"
def _boom(*a, **k):
async def gen():
raise RuntimeError("上游炸了")
yield b"" # pragma: no cover
return gen()
monkeypatch.setattr(R, "upstream_stream", _boom)
assert asyncio.run(_grade_answer(e, "问", "答", "cloud-x", 5)) == "ERROR"
assert calls["n"] == 2
def test_shadow_review_task_feeds_promotion(tmp_path, monkeypatch):
"""PASS/FAIL 写入晋升表(observe 计数),ERROR 只计错误不进通过率。"""
from gateway.sense.store import SenseStore
sstore = SenseStore.init_db(str(tmp_path / "sense.sqlite3"))
promo = PromotionTable(sstore, now=lambda: 1_000_000.0)
label = promotion_label("proxy", "")
fake, _ = _fake_upstream(["PASS", "PASS", "FAIL", "垃圾输出"])
monkeypatch.setattr(R, "upstream_stream", fake)
entry = _entry()
sd = {"sense": {"shadow_review": {"enabled": True, "sample_rate": 1.0}}}
async def _run():
for _ in range(4):
await R._shadow_review_task("问题", "本地答案", _FixedPool(entry),
sd, promo)
asyncio.run(_run())
assert R._shadow_review_stats == {"sampled": 4, "pass": 2, "fail": 1,
"error": 1, "skipped_busy": 0}
row = sstore.get_promotion(label)
assert row["n_total"] == 3 and row["n_ok"] == 2 # ERROR 不进通过率
class _FixedPool:
def __init__(self, entries):
self._e = entries if isinstance(entries, list) else [entries]
def usable_entries(self):
return list(self._e)
def test_maybe_shadow_review_disabled_is_noop(monkeypatch):
"""默认关闭:不建任务、不计采样。"""
class _SP:
@staticmethod
def to_dict():
return {"sense": {"shadow_review": {"enabled": False}}}
payload = json.dumps({"choices": [{"message": {"content": "本地答案"}}]}).encode()
resp = type("JR", (), {"body": payload})()
before = dict(R._shadow_review_stats)
R._maybe_shadow_review({}, resp, _FixedPool(_entry()), _SP, scfg=None)
assert R._shadow_review_stats == before
+70
View File
@@ -0,0 +1,70 @@
"""TPM 双桶限流与缓存命中率透出(T-X12,采纳 enterprise-ai-gateway 设计)。"""
import time
from gateway.proxy.auth import RateLimiter
from gateway.proxy.semcache import SemanticCache
def test_tpm_bucket_capacity_and_consume():
"""容量内放行并扣减;超过余量拒绝。"""
rl = RateLimiter()
assert rl.allow_tokens(1, 100, 1000) is True
assert rl.allow_tokens(1, 500, 1000) is True
# 已扣 600,剩 400 + 回流(瞬时近似 0):700 > 400+eps -> 拒
assert rl.allow_tokens(1, 700, 1000) is False
def test_tpm_bucket_continuous_refill(monkeypatch):
"""连续时间回流:elapsed * tpm/60 补充(容量封顶)。"""
rl = RateLimiter()
clock = {"t": 100.0}
monkeypatch.setattr(time, "monotonic", lambda: clock["t"])
assert rl.allow_tokens(2, 1000, 2000) is True # 剩 1000
clock["t"] = 130.0 # 30s 回流 1000 -> 满 2000
assert rl.allow_tokens(2, 1500, 2000) is True # 扣 1500 剩 500
clock["t"] = 145.0 # 15s 回流 500 -> 1000
assert rl.allow_tokens(2, 1000, 2000) is True # 精确扣空
assert rl.allow_tokens(2, 1, 2000) is False # 空桶拒绝
def test_tpm_disabled_when_cap_zero():
"""tpm_cap <= 0 = 不限流(恒放行)。"""
rl = RateLimiter()
assert rl.allow_tokens(3, 10**9, 0) is True
def test_tpm_independent_from_rpm_bucket():
"""双桶相互独立:打满 TPM 不影响 RPM 放行,RPM 打空不影响 TPM 记账。"""
rl = RateLimiter()
assert rl.allow_tokens(4, 5000, 5000) is True # TPM 打空
assert rl.allow(4, 10) is True # RPM 仍放行
for _ in range(9): # 共 10 次 = 打满 RPM
assert rl.allow(4, 10) is True
assert rl.allow(4, 10) is False # RPM 空桶
assert rl.allow_tokens(4, 1, 5000) is False # TPM 仍空(未被 RPM 动过)
def test_tpm_per_key_isolation():
"""按 key 隔离:一个 key 打满不影响另一个。"""
rl = RateLimiter()
assert rl.allow_tokens(5, 5000, 5000) is True
assert rl.allow_tokens(6, 5000, 5000) is True
assert rl.allow_tokens(5, 5000, 5000) is False
def test_semcache_stats_hit_rate():
"""stats 透出 misses 与 hit_rate(命中率口径 = hits / (hits + misses))。"""
class _S:
def semcache_rows(self, limit=0):
return []
def put_semcache(self, *a, **k):
pass
c = SemanticCache(_S(), max_entries=10, sim_threshold=0.5, promote_frequency=5)
c.put("k1", "请解释一下递归函数的概念", "答案", "m", doc_version=1)
assert c.lookup("k1", "请解释一下递归函数的概念", doc_version=1) is not None
assert c.lookup("k9", "完全无关的问题内容呢", doc_version=1) is None
s = c.stats()
assert s["hits_exact"] == 1 and s["misses"] == 1
assert s["hit_rate"] == 0.5
@@ -1,14 +1,14 @@
{ {
"schemaVersion": "mimosa-hook-status/v1", "schemaVersion": "mimosa-hook-status/v1",
"recordedAt": "2026-09-05T07:03:44.339Z", "recordedAt": "2026-09-05T08:17:05.111Z",
"sessionId": "sess_e50d4f25-3ac6-43f2-b2f3-8833b3150465", "sessionId": "sess_e50d4f25-3ac6-43f2-b2f3-8833b3150465",
"event": "PostToolUse", "event": "PostToolUse",
"toolName": "Edit", "toolName": "Edit",
"file": "src/views/MetricsView.vue", "file": "src/App.vue",
"outcome": "clear", "outcome": "clear",
"coverage": "complete", "coverage": "complete",
"findingCount": 0, "findingCount": 0,
"durationMs": 2, "durationMs": 3,
"hostState": "hook_complete", "hostState": "hook_complete",
"reportHint": ".mimosa/reports/" "reportHint": ".mimosa/reports/"
} }
+1
View File
@@ -38,6 +38,7 @@ const NAV = [
{ to: '/agent', icon: '🤖', label: '智能体' }, { to: '/agent', icon: '🤖', label: '智能体' },
{ to: '/review', icon: '🔍', label: '人工检验' }, { to: '/review', icon: '🔍', label: '人工检验' },
{ to: '/metrics', icon: '📊', label: '指标' }, { to: '/metrics', icon: '📊', label: '指标' },
{ to: '/proxy', icon: '🏷️', label: '代理' },
{ to: '/settings', icon: '⚙️', label: '设置' }, { to: '/settings', icon: '⚙️', label: '设置' },
] ]
</script> </script>
+81
View File
@@ -543,6 +543,87 @@ export function watchAgent(requestId: string) {
} }
} }
// ── 代理层管理面(T-P7) ─────────────────────────────────────────────────────
export interface ProxyStats {
requests: number
h_g: number
h_p: number
revenue_milli: number
cost_milli: number
margin_milli: number
by_bucket?: Record<string, { requests: number; revenue_milli: number; cost_milli: number }>
}
export interface ProxyLedgerRow {
request_id: string
ts: number
model: string
bucket: string
charged_milli: number
upstream_cost_milli: number
margin_milli: number
status: string
}
export interface ProxyStudent {
id: number
name: string
class: string
status: string
balance_milli: number
}
/** GET /proxy/admin/stats:命中率-毛利看板 */
export async function getProxyStats() {
const { data } = await http.get<ProxyStats>('/proxy/admin/stats')
return data
}
/** GET /proxy/admin/ledger:流水分页 */
export async function getProxyLedger(studentId = 0, limit = 50, offset = 0) {
const { data } = await http.get<ProxyLedgerRow[]>('/proxy/admin/ledger',
{ params: { student_id: studentId || undefined, limit, offset } })
return data
}
/** GET /proxy/admin/students:学生列表(后端暂 501 时返回空) */
export async function listProxyStudents() {
try {
const { data } = await http.get<ProxyStudent[]>('/proxy/admin/students')
return data
} catch {
return [] as ProxyStudent[]
}
}
/** POST /proxy/admin/students:建学生 */
export async function createProxyStudent(name: string, balanceYuan: number, klass = '') {
const { data } = await http.post('/proxy/admin/students',
{ name, class: klass, balance_yuan: balanceYuan })
return data as unknown as ProxyStudent & { student_id: number }
}
/** POST /proxy/admin/students/{id}/topup:充值 */
export async function topupProxyStudent(id: number, amountYuan: number) {
const { data } = await http.post<{ ok: boolean; balance_milli: number }>(
`/proxy/admin/students/${id}/topup`, { amount_yuan: amountYuan })
return data
}
/** POST /proxy/admin/keys:签发 key(明文只返回一次) */
export async function issueProxyKey(studentId: number) {
const { data } = await http.post<{ key_id: number; key: string; prefix: string }>(
'/proxy/admin/keys', { student_id: studentId })
return data
}
/** POST /proxy/admin/keys/{id}/revoke:注销 */
export async function revokeProxyKey(keyId: number) {
const { data } = await http.post<{ ok: boolean }>(`/proxy/admin/keys/${keyId}/revoke`)
return data
}
// ── 辅助:轮询直到完成(用于不需要 SSE 的场景)────────────────────────────────── // ── 辅助:轮询直到完成(用于不需要 SSE 的场景)──────────────────────────────────
export async function pollUntilDone( export async function pollUntilDone(
+5
View File
@@ -36,6 +36,11 @@ const router = createRouter({
name: 'metrics', name: 'metrics',
component: () => import('@/views/MetricsView.vue'), component: () => import('@/views/MetricsView.vue'),
}, },
{
path: '/proxy',
name: 'proxy',
component: () => import('@/views/ProxyView.vue'),
},
{ {
path: '/settings', path: '/settings',
name: 'settings', name: 'settings',
+229
View File
@@ -0,0 +1,229 @@
<template>
<div class="proxy-view">
<header class="page-head">
<h2>🏷️ 校园代理管理</h2>
<p class="sub">学生 key / 用量流水 / 命中率-毛利看板(毫元计费,1 元 = 1000 毫元)</p>
</header>
<div v-if="error" class="error">{{ error }}</div>
<!-- 卡1:命中率-毛利看板 -->
<div class="card-grid" v-if="stats">
<div class="metric-card highlight">
<h3>缓存命中率</h3>
<div class="kv-list">
<span>网关直答 h_g</span><b>{{ pct(stats.h_g) }}</b>
<span>上游前缀 h_p</span><b>{{ pct(stats.h_p) }}</b>
<span>综合(近似)</span><b>{{ pct(stats.h_g + stats.h_p) }}</b>
</div>
<p class="hint">北极星:h = h_g + h_p;目标 ≥50%</p>
</div>
<div class="metric-card">
<h3>毛利(毫元)</h3>
<div class="kv-list">
<span>收入 Σcharged</span><b>{{ stats.revenue_milli }}</b>
<span>成本 Σcost</span><b>{{ stats.cost_milli }}</b>
<span>毛利</span>
<b :class="stats.margin_milli >= 0 ? 'pos' : 'neg'">{{ stats.margin_milli }}</b>
<span>请求数</span><b>{{ stats.requests }}</b>
</div>
</div>
<div class="metric-card" v-if="buckets.length">
<h3>分桶</h3>
<div class="kv-list">
<template v-for="b in buckets" :key="b.name">
<span>{{ b.name }}</span>
<b>{{ b.requests }} 次 / {{ b.revenue_milli }} 毫元</b>
</template>
</div>
</div>
</div>
<!-- 卡2:key 管理 -->
<div class="metric-card">
<h3>学生与 Key 管理</h3>
<div class="issue-row">
<input v-model="newName" placeholder="学生姓名" class="sm" />
<input v-model="newBalance" placeholder="初始余额(元)" class="sm num" />
<button class="btn" @click="addStudent">建学生</button>
<button class="btn" :disabled="!selectedStudent" @click="issue">签发 Key</button>
<button class="btn" :disabled="!selectedStudent" @click="topup">充值 1 元</button>
</div>
<div v-if="issuedKey" class="issued">
⚠️ 新 key(仅显示一次,请复制): <code>{{ issuedKey }}</code>
</div>
<table class="tbl" v-if="students.length">
<thead>
<tr><th>ID</th><th>姓名</th><th>余额(毫元)</th><th>状态</th><th>操作</th></tr>
</thead>
<tbody>
<tr v-for="s in students" :key="s.id" :class="{ sel: s.id === selectedStudent }"
@click="selectedStudent = s.id">
<td>{{ s.id }}</td><td>{{ s.name }}</td><td>{{ s.balance_milli }}</td>
<td>{{ s.status }}</td>
<td><button class="mini" @click.stop="selectedStudent = s.id; issue()">签发</button></td>
</tr>
</tbody>
</table>
<p v-else class="hint">暂无学生(后端列表接口待补时,可用上方"建学生"先创建)</p>
</div>
<!-- 卡3:用量流水 -->
<div class="metric-card">
<h3>用量流水</h3>
<table class="tbl" v-if="ledger.length">
<thead>
<tr><th>时间</th><th>模型</th><th>状态</th>
<th>收入</th><th>成本</th><th>毛利</th></tr>
</thead>
<tbody>
<tr v-for="r in ledger" :key="r.request_id">
<td class="mono">{{ fmtTime(r.ts) }}</td>
<td class="mono">{{ r.model }}</td>
<td><span :class="['st', r.status]">{{ r.status }}</span></td>
<td>{{ r.charged_milli }}</td>
<td>{{ r.upstream_cost_milli }}</td>
<td>{{ r.margin_milli }}</td>
</tr>
</tbody>
</table>
<p v-else class="hint">暂无流水</p>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, computed, onMounted } from 'vue'
import {
getProxyStats, getProxyLedger, listProxyStudents,
createProxyStudent, topupProxyStudent, issueProxyKey,
} from '@/api'
import type { ProxyStats, ProxyLedgerRow, ProxyStudent } from '@/api'
const stats = ref<ProxyStats | null>(null)
const ledger = ref<ProxyLedgerRow[]>([])
const students = ref<ProxyStudent[]>([])
const selectedStudent = ref<number>(0)
const newName = ref('')
const newBalance = ref('1')
const issuedKey = ref('')
const error = ref('')
const buckets = computed(() => {
const bb = stats.value?.by_bucket || {}
return Object.entries(bb).map(([name, v]) => ({ name, ...v }))
})
function pct(v: number) { return (v * 100).toFixed(1) + '%' }
function fmtTime(ts: number) {
const d = new Date(ts * 1000)
return `${d.getMonth() + 1}/${d.getDate()} ${String(d.getHours()).padStart(2, '0')}:${String(d.getMinutes()).padStart(2, '0')}`
}
async function load() {
error.value = ''
try {
const [s, l, st] = await Promise.all([
getProxyStats(), getProxyLedger(0, 50, 0), listProxyStudents()])
stats.value = s
ledger.value = l
students.value = st
} catch (e: any) {
error.value = e?.response?.data?.detail || e?.message || String(e)
}
}
async function addStudent() {
if (!newName.value.trim()) return
try {
const r = await createProxyStudent(newName.value.trim(), Number(newBalance.value) || 0)
selectedStudent.value = (r as any).student_id || 0
newName.value = ''
issuedKey.value = ''
await load()
} catch (e: any) { error.value = e?.message || String(e) }
}
async function issue() {
if (!selectedStudent.value) return
try {
const r = await issueProxyKey(selectedStudent.value)
issuedKey.value = r.key
await load()
} catch (e: any) { error.value = e?.response?.data?.detail || e?.message || String(e) }
}
async function topup() {
if (!selectedStudent.value) return
try {
await topupProxyStudent(selectedStudent.value, 1)
await load()
} catch (e: any) { error.value = e?.message || String(e) }
}
onMounted(load)
</script>
<style scoped>
.proxy-view { padding: 20px 24px; height: 100%; overflow-y: auto; }
.page-head { display: flex; flex-direction: column; margin-bottom: 16px; }
.sub { color: var(--c-text-2); font-size: 13px; margin-top: 2px; }
.error { color: var(--c-err); margin-bottom: 10px; }
.card-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
gap: 16px;
margin-bottom: 16px;
}
.metric-card {
background: var(--c-surface);
border: 1px solid var(--c-border-soft);
border-radius: var(--radius);
padding: 14px 16px;
box-shadow: var(--shadow-card);
}
.metric-card h3 { font-size: 14px; color: var(--c-text); margin-bottom: 10px; }
.kv-list { display: grid; grid-template-columns: 1fr 1fr; gap: 6px 10px; font-size: 13px; }
.kv-list span { color: var(--c-text-2); }
.pos { color: var(--c-ok); }
.neg { color: var(--c-err); }
.hint { color: var(--c-caption); font-size: 11px; margin-top: 8px; }
.issue-row { display: flex; gap: 8px; margin-bottom: 10px; flex-wrap: wrap; }
.issue-row .sm {
border: 1px solid var(--c-border);
border-radius: 6px;
padding: 6px 10px;
font-size: 13px;
}
.issue-row .num { width: 110px; }
.btn {
border: 1px solid var(--c-border);
background: var(--c-surface);
border-radius: 6px;
padding: 6px 12px;
font-size: 13px;
cursor: pointer;
}
.btn:hover:not(:disabled) { border-color: var(--c-primary); color: var(--c-primary); }
.btn:disabled { opacity: 0.5; cursor: not-allowed; }
.issued {
background: var(--c-warn-soft);
border: 1px solid var(--c-warn);
border-radius: 6px;
padding: 8px 10px;
font-size: 12px;
margin-bottom: 10px;
word-break: break-all;
}
.tbl { width: 100%; border-collapse: collapse; font-size: 12.5px; }
.tbl th, .tbl td { text-align: left; padding: 6px 8px; border-bottom: 1px solid var(--c-border-soft); }
.tbl th { color: var(--c-text-2); font-weight: 600; background: var(--ds-neutral-bluish-50); }
.tbl tr.sel { background: var(--c-primary-soft); }
.mono { font-family: var(--font-mono); font-size: 11.5px; }
.st { font-size: 11px; padding: 1px 6px; border-radius: 8px; background: var(--ds-neutral-bluish-100); }
.st.cached { background: var(--c-ok-soft); color: var(--c-ok); }
.st.failed, .st.aborted, .st.insufficient { background: var(--c-err-soft); color: var(--c-err); }
.mini { border: 1px solid var(--c-border); background: var(--c-surface); border-radius: 5px;
padding: 2px 8px; font-size: 11px; cursor: pointer; }
.mini:hover { border-color: var(--c-primary); color: var(--c-primary); }
</style>
+11 -3
View File
@@ -141,9 +141,9 @@ P0 完成后的能力:干净的后端抽象 + 可量化的评测 + 可追溯
| T-P3 | 计价+结算:峰谷窗口/毫元整数/黄金用例 ≥10 组 | ✅ 完成 | T-P3 | | T-P3 | 计价+结算:峰谷窗口/毫元整数/黄金用例 ≥10 组 | ✅ 完成 | T-P3 |
| T-P4 | 路由端到端:/proxy/v1 非流式+流式+402/429/413 语义 | ✅ 完成 | T-P4 | | T-P4 | 路由端到端:/proxy/v1 非流式+流式+402/429/413 语义 | ✅ 完成 | T-P4 |
| T-P5 | 规范化+桶:稳定哈希/整形顺序/doc_version 失效/多轮不缓存 | ✅ 完成 | T-P5 | | T-P5 | 规范化+桶:稳定哈希/整形顺序/doc_version 失效/多轮不缓存 | ✅ 完成 | T-P5 |
| T-P6 | 语义缓存:L1 精确+L2 n-gram 倒排+singleflight+TTL+失败不缓存 | ⬜ 待办 | | | T-P6 | 语义缓存:L1 精确+L2 n-gram 倒排+singleflight+TTL+失败不缓存 | ✅ 完成(含 routes 接线) | T-P6 |
| T-P7 | 管理面+前端:stats/ledger 端点+ProxyView 三卡片 | ⬜ 待办 | | | T-P7 | 管理面+前端:stats/ledger 端点+ProxyView 三卡片 | ✅ 完成(students 列表端点 501 待扫描误报解除) | T-P7 |
| T-P8 | 压测+预算:200 并发流式;P99≤50ms/内存≤1GB/账目零不一致 | ⬜ 待办 | | | T-P8 | 压测+预算:200 并发流式;P99≤50ms/内存≤1GB/账目零不一致 | ✅ 完成(mock 管道 h_g=100%/P99=23.5ms/吞吐 205rps;--live 桩就绪待真实 key) | T-P8 |
--- ---
@@ -164,3 +164,11 @@ P0 完成后的能力:干净的后端抽象 + 可量化的评测 + 可追溯
| T-G6 | live 分流:pipeline tier_fn 三档钩子 + proxy 档位映射 + T2 档升级阶梯 | ✅ 完成 | T-G6 | | T-G6 | live 分流:pipeline tier_fn 三档钩子 + proxy 档位映射 + T2 档升级阶梯 | ✅ 完成 | T-G6 |
| T-G7 | 审计+前端:ReviewQueue 抽样 + tier 指标卡 + 客户端来源显示/一键升级 | ✅ 完成 | T-G7 | | T-G7 | 审计+前端:ReviewQueue 抽样 + tier 指标卡 + 客户端来源显示/一键升级 | ✅ 完成 | T-G7 |
| T-G8 | 实验:E-G1/E-G3 报告;(可选)LoraRemote + E-G2 线性 vs LoRA | ✅ 完成 | T-G8 | | T-G8 | 实验:E-G1/E-G3 报告;(可选)LoraRemote + E-G2 线性 vs LoRA | ✅ 完成 | T-G8 |
| OPT-1 | 分支推进:语义缓存 L2 查找 3.39x(免并集计分+预筛)+ 安全加固(15 高危清零:SSRF/路径穿越/假凭据) | ✅ 完成 | ad3bf41 |
| T-X1 | 预算四档渐进干预(外部采纳 ai-model-router):budget_mode 整数基点判定(80/95/100%)+ optimize/cheap 自动降档 + X-Budget-Mode 上报;黄金用例锁边界 | ✅ 完成 | T-X1 |
| T-X2 | 上游有序降级链(外部采纳 cortiq tier 链):池内候选按档位/单价排序、首 token 前 failover、X-Upstream-Fallback 三元组响应头 + admin/stats failover 聚合 | ✅ 完成 | T-X2 |
| T-X3 | 路由决策缓存(外部采纳 cortiq 决策哈希缓存):DecisionCache(sha256+LRU+TTL60s,4096 条)前置 grader live 模式,embed+线性头去重;观察/审计不跳过、invalidate 同步清空、collect/shadow 不缓存 | ✅ 完成 | T-X3 |
| T-X6 | 模型池能力位过滤(外部采纳 cortiq capabilities):条目 capabilities{vision,tools,context_window}(缺省全兼容)+ filter_by_capabilities 硬过滤 + 代理请求需求推断(图片→vision/tools/上下文)与重定向 | ✅ 完成 | T-X6 |
| T-X4 | 可解释决策留痕(外部采纳 ai-model-router 决策三元):sense_decisions 表(reasons/candidate_scores/rejected)+ grader 全模式落库 + /proxy/admin/sense-decisions 查询;附带修复 settings_provider 未注入导致 sense live 分流失效的潜伏 bug | ✅ 完成 | T-X4 |
| T-X5 | 云端评判晋升表×conformal 双闸门(外部采纳 cortiq promotion):PromotionTable 状态机(n_min/通过率/soak 晋升、退化降级、T3 一票否决)+ grader live 模式 T2→T1 应用 + 人工核验 verdict 回填(promo 标签) | ✅ 完成 | T-X5 |
| T-X7~X9 | P2 立项不实施:多协议透传(/v1/messages)/ 工具输出清洗沙箱(受限 DSL)/ 网关内置 A/B 实验框架 | 📋 立项 | 待排期 |
+11
View File
@@ -206,3 +206,14 @@
2. 真机项:llama-server embedder 端点冒烟(需 bge-m3 模型);DeepSeek key 重配后 2. 真机项:llama-server embedder 端点冒烟(需 bge-m3 模型);DeepSeek key 重配后
走 collect 模式积累真实观察(2-4 周)→ sense_report 晋升门核对 → live。 走 collect 模式积累真实观察(2-4 周)→ sense_report 晋升门核对 → live。
3. 备份锚点:`sense-m-g3` tag(412 全绿)。 3. 备份锚点:`sense-m-g3` tag(412 全绿)。
### 6.4 增补(同日):代理层 T-P6~T-P8 完成(M2+M3 达成)
- T-P6 语义缓存(e41471c + c54ad23 接线):L1/L2 倒排+晋升+singleflight+SSE 回放;
e2e 实证同问二答 X-Cache: HIT 上游仅 1 次、账目 status=cached cost=0。
- T-P7 管理面(4f272dc + 7caab52):stats/ledger 端点 + ProxyView 三卡片
(students 列表端点 501——Mimosa 扫描误报阻塞新 SELECT 写入,解除后补两方法)。
- T-P8 压测(bench_proxy.py):mock 管道 h_g=100%(重复>=50% 数据集)/
P99=23.5ms(预算 50ms)/吞吐 205 req/s/账目零不一致;--live 桩就绪
(CAMPUS_PROXY_KEY 环境变量 + --yes 确认)。
- **M3 总验收达成**(mock 口径);报告落盘 AI代理功能开发/bench/。
- 回档锚点:sense-m-g3 之上新增 proxy-m2-m3 tag(423 全绿)。
+95
View File
@@ -0,0 +1,95 @@
# 科研技能清单(计算机科学与技术方向)
> 已安装 [Scientific Agent Skills](https://github.com/K-Dense-AI/scientific-agent-skills)(163 个)至用户级技能目录,重启 ZCode 后生效。
> 本清单筛选出 **CS 科研常用**与**通用**技能,标注⭐为本毕设(端云协同编程智能体系统)直接可用。
> 未列出的约 100 个为生物/化学/医药/量子等领域专用技能,已安装但在 CS 方向大概率不会触发。
---
## 一、文献与调研(选题/相关工作/开题)
| 技能 | 用途 | 备注 |
|---|---|---|
| literature-review | 多学术数据库系统性文献综述 | ⭐ 相关工作章节 |
| paper-lookup | 11 个学术 API 检索论文/预印本/引用 | ⭐ 快速找论文 |
| research-lookup | 为手稿汇编当前学术证据 | 写作时引用支撑 |
| citation-management | OpenAlex/PubMed/Google Scholar 引文管理 | ⭐ 参考文献格式 |
| bgpt-paper-search | 论文检索 + 全文实验数据提取 | |
| exa-search | 面向科学/技术内容的网络检索 | 需 Exa key |
| database-lookup | 公共数据库 API 规范查询 | |
## 二、实验设计与数据(评测/实验章节)
| 技能 | 用途 | 备注 |
|---|---|---|
| experimental-design | 数据收集前的实验设计(随机化/区组/样本量) | ⭐ 评测实验设计 |
| statistical-analysis | 统计检验选择/假设检验/效应量/APA 报告 | ⭐ 对比实验分析 |
| statistical-power | 样本量与统计功效计算 | 判断 200 条数据集是否够 |
| exploratory-data-analysis | 有界探索性数据分析(CSV/表格) | ⭐ 跑分数据初探 |
| scientific-critical-thinking | 评估实验设计与证据质量 | ⭐ 审自己的评测结论 |
| hypothesis-generation | 证据约束的科学假设生成 | |
## 三、机器学习与智能体(系统核心相关)
| 技能 | 用途 | 备注 |
|---|---|---|
| scikit-learn | 监督/无监督 ML 基线 | ⭐ 分类器/缓存预测基线 |
| pytorch-lightning | PyTorch 工程化训练 | ⭐ tier 线性头升级时 |
| transformers | HF 模型加载/推理 | ⭐ 本地小模型 |
| shap | ML 预测解释(SHAP) | ⭐ 分级决策可解释性 |
| stable-baselines3 | 强化学习算法(PPO/SAC/DQN) | agent 相关研究 |
| torch-geometric | 图神经网络 | 依赖图建模 |
| hugging-science | 科研领域 AI/ML 工作流 | |
| timesfm-forecasting | 时间序列零样本预测 | 负载预测类扩展 |
## 四、写作与呈现(论文/答辩)
| 技能 | 用途 | 备注 |
|---|---|---|
| scientific-writing | 论文草稿/修订/证据链审计 | ⭐ 毕业论文主体 |
| markdown-mermaid-writing | Markdown + Mermaid 架构图/流程图 | ⭐ 系统架构图 |
| scientific-visualization | 出版级科研图表(真实、无误导) | ⭐ 实验图表 |
| scientific-slides | 答辩/组会幻灯片 | ⭐ 答辩 PPT |
| scientific-schematics | AI 生成科研示意图 | 系统示意图 |
| peer-review | 同行评审级稿件评估 | ⭐ 交稿前自审 |
| venue-templates | 期刊/会议/海报模板 | |
| latex-posters | LaTeX 学术海报 | |
## 五、通用工具(日常科研)
| 技能 | 用途 | 备注 |
|---|---|---|
| pdf | PDF 读取/拆分/合并/提取 | ⭐ 读文献 PDF |
| markitdown / liteparse | 文档/PDF → Markdown | ⭐ 喂给 LLM 前的格式转换 |
| docx / xlsx / pptx | Office 三件套读写 | ⭐ 任务书/中期表 |
| matplotlib / seaborn | 绘图基础库 | ⭐ |
| sympy | 符号数学(公式推导验证) | 复杂度分析 |
| networkx | 图/网络分析建模 | |
| polars / dask | 大表数据处理/分布式 | 评测数据量大时 |
| infographics | AI 信息图生成 | 汇报配图 |
| generate-image | AI 生图/改图 | |
| what-if-oracle | 结构化 What-If 情景分析 | 方案论证 |
| scientific-brainstorming | 证据感知的科学头脑风暴 | 选题/方案发散 |
## 六、系统/环境(需要时)
| 技能 | 用途 |
|---|---|
| optimize-for-gpu | NVIDIA GPU 加速科学 Python(一致性校验) |
| modal | 无服务器云算力跑 Python(重负载时) |
| get-available-resources | 探测宿主机 CPU/内存/磁盘/GPU 清单 |
| matlab | MATLAB/Octave 数值工作流(如课程需要) |
---
## 未列入的方向专用技能(已安装,按需查阅)
生物信息(biopython/scanpy/pysam 等 ~40 个)、化学与药物(rdkit/deepchem/medchem 等 ~20 个)、
临床与医疗(pyhealth/clinical-* 等 ~15 个)、量子计算(qiskit/pennylane/qutip)、
地理(geopandas/geomaster)、实验室自动化(pylabrobot/opentrons)等。
完整清单:`~/.agents/skills/` 目录,或用 `find-skills` 技能按需检索。
## 生效说明
技能在 **ZCode 会话启动时**扫描加载——重启会话后即可在对话中直接触发
(如"用 statistical-analysis 分析这份评测数据"),无需手动安装依赖(用到时按技能内指引装)。