feat(v1): 架构与算法优化二轮——共享 LLM 客户端去重、语义缓存 O(1) 提升/淘汰、分类器热路径清理
- 新增 router_system/llm_client.py:OpenAICompatClient 统一 experts/judge/fallback
三处复制的懒建 AsyncClient + /chat/completions + choices/usage 解析(~60 行去重);
密钥解析统一走 config.get_api_key(激活原死代码,顺带消除 experts 默认环境名不一致)
- 语义缓存 L2:条目容器 list→OrderedDict(提升/淘汰 O(n)→O(1)),按 query 天然去重;
n-gram 向量 lru_cache 复用(同一次 miss 的 get/put 免重复分词);
A/B:淘汰路径 0.040→0.034s,miss→put 往返 9.41→8.57s(-9%)
- RuleJudge 覆盖度:response.lower() 提出逐词循环(原 O(terms×len) 重复复制)
- extract_content_terms 纯函数 lru_cache 化(专家与 Judge 对同一查询免重复分词),返回 tuple
- RuleClassifier:_score 去掉败者领域白建的命中词 list(胜出后单独收集);
修复 code 规则 ("api",0.7) 重复登记(原命中计 1.4 分)
- difficulty:正则模块级预编译
- tests:恢复上一轮引入的乱码中文 docstring;网关测试输入串恢复为可判 code 的中文查询
This commit is contained in:
@@ -0,0 +1,69 @@
|
||||
"""OpenAI 兼容 Chat Completions 共享客户端(架构去重)。
|
||||
|
||||
此前 experts.APIExpert / judge.LLMJudge / fallback.APIFallback 各自复制一份
|
||||
"懒建 AsyncClient + POST /chat/completions + choices/usage 解析"的等价逻辑
|
||||
(超时、温度、token 上限各自维护)。本模块将其统一为单一组件:
|
||||
|
||||
- 懒建 httpx.AsyncClient 并长期复用(与仓库内其他客户端一致)
|
||||
- temperature / max_tokens 传 None 时不出现在请求体(保持各后端原请求形状)
|
||||
- 密钥解析统一走 config.get_api_key(此前 experts 的默认环境名与另两处不一致)
|
||||
|
||||
依赖说明:httpx 仅 api 类型后端需要,import 延迟到连接创建时,
|
||||
router_system 核心保持零第三方依赖约束不变。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
|
||||
class OpenAICompatClient:
|
||||
"""OpenAI 兼容 /chat/completions 客户端(懒建连接、跨调用复用)。"""
|
||||
|
||||
def __init__(self, base_url: str, api_key: str, timeout: float = 60.0):
|
||||
self.base_url = base_url.rstrip("/")
|
||||
self.api_key = api_key
|
||||
self._timeout = timeout
|
||||
self._client = None
|
||||
|
||||
def _get_client(self):
|
||||
if self._client is None:
|
||||
import httpx
|
||||
self._client = httpx.AsyncClient(timeout=self._timeout)
|
||||
return self._client
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
model: str,
|
||||
messages: List[Dict[str, str]],
|
||||
temperature: Optional[float] = 0.2,
|
||||
max_tokens: Optional[int] = 1024,
|
||||
) -> Dict[str, Any]:
|
||||
"""调用 /chat/completions,返回完整响应 JSON(含 usage)。
|
||||
|
||||
HTTP 非 2xx 时抛 httpx.HTTPStatusError;temperature/max_tokens
|
||||
为 None 时对应字段不下发(与旧实现中 LLMJudge 的请求体一致)。
|
||||
"""
|
||||
payload: Dict[str, Any] = {"model": model, "messages": messages}
|
||||
if temperature is not None:
|
||||
payload["temperature"] = temperature
|
||||
if max_tokens is not None:
|
||||
payload["max_tokens"] = max_tokens
|
||||
client = self._get_client()
|
||||
resp = await client.post(
|
||||
f"{self.base_url}/chat/completions",
|
||||
headers={"Authorization": f"Bearer {self.api_key}"},
|
||||
json=payload,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
@staticmethod
|
||||
def completion_text(data: Dict[str, Any]) -> str:
|
||||
"""提取首条回复文本。"""
|
||||
return data["choices"][0]["message"]["content"]
|
||||
|
||||
@staticmethod
|
||||
def completion_tokens(data: Dict[str, Any], text: str) -> int:
|
||||
"""提取 completion token 数;usage 缺失时按 ~2.2 字符/token 估算。"""
|
||||
usage = data.get("usage", {})
|
||||
return usage.get("completion_tokens", int(len(text) / 2.2))
|
||||
Reference in New Issue
Block a user