fix(openai): strict OpenAI wire compatibility — Open WebUI can chat again
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s

/v1/chat/completions answered ordinary client payloads with a bare HTTP 422
(a plain-text body no OpenAI client can parse), so Open WebUI showed neither
an answer nor an error — it just hung. Parse the body leniently instead:
content as string/null/content-part array, tools: null, stream: null, a
string reasoning_effort, max_completion_tokens, messages[].name and
tool_call_id, and a missing Content-Type. Unreadable bodies now come back as
an OpenAI error envelope with 400.

Responses are strict OpenAI JSON: no "tool_calls": null, finish_reason mapped
to the spec's closed set (cortiq serve's max_tokens -> length), never an empty
id, and `model` echoes the id the client requested — the model that actually
answered stays in cortiq.selected_model and X-Cortiq-Selected-Model.

Streaming is re-normalised rather than forwarded verbatim: a leading role
delta, message-shaped chunks converted to delta, one terminal finish_reason,
always a trailing data: [DONE], and an SSE error event when the upstream dies
mid-flight. Adds X-Accel-Buffering: no so a fronting proxy cannot buffer the
stream into a hang.

GET /v1/models now lists managed [[cmf.servers]] models with their live
running/healthy state, so clients can pin a local .cmf directly.

Also strips client bookkeeping keys (chat_id, session_id, metadata, features,
...) from the upstream call and drops the leftover [to_wire] debug prints.
This commit is contained in:
infosave2007
2026-08-21 09:10:31 +03:00
parent 34093330f6
commit b339c2b622
15 changed files with 660 additions and 93 deletions
+42 -2
View File
@@ -15,13 +15,53 @@ pub enum RoutingDirective {
Pinned { model_id: String },
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
pub struct Message {
pub role: String, // system | user | assistant | tool
#[serde(default)]
/// Always a plain string internally. On the wire OpenAI clients legitimately
/// send `null` (assistant turns that only carry tool calls) or an array of
/// content parts (Open WebUI and every other multimodal UI), so
/// [`de_content`] flattens both shapes instead of rejecting the request.
#[serde(default, deserialize_with = "de_content")]
pub content: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub tool_calls: Vec<serde_json::Value>,
/// Speaker name (`messages[].name`) — proxied through when the client sends it.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
/// Set on `role = "tool"` replies; dropping it breaks multi-turn tool calling.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_call_id: Option<String>,
}
/// Deserialize `messages[].content` from any shape an OpenAI-compatible client
/// may send: a string, `null`, or an array of content parts. Text parts are
/// concatenated; non-text parts (images, audio, files) are skipped — this
/// gateway routes text, and a picture must not turn into a 422.
fn de_content<'de, D>(d: D) -> std::result::Result<String, D::Error>
where
D: serde::Deserializer<'de>,
{
use serde::Deserialize as _;
Ok(match serde_json::Value::deserialize(d)? {
serde_json::Value::Null => String::new(),
serde_json::Value::String(s) => s,
serde_json::Value::Array(parts) => {
let mut out = String::new();
for part in parts {
// {"type":"text","text":"..."} / {"type":"input_text","text":"..."}
let text = part
.get("text")
.and_then(|t| t.as_str())
.or_else(|| part.as_str());
if let Some(t) = text {
out.push_str(t);
}
}
out
}
other => other.to_string(),
})
}
#[derive(Clone, Debug, Default)]