fix(openai): strict OpenAI wire compatibility — Open WebUI can chat again
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
/v1/chat/completions answered ordinary client payloads with a bare HTTP 422 (a plain-text body no OpenAI client can parse), so Open WebUI showed neither an answer nor an error — it just hung. Parse the body leniently instead: content as string/null/content-part array, tools: null, stream: null, a string reasoning_effort, max_completion_tokens, messages[].name and tool_call_id, and a missing Content-Type. Unreadable bodies now come back as an OpenAI error envelope with 400. Responses are strict OpenAI JSON: no "tool_calls": null, finish_reason mapped to the spec's closed set (cortiq serve's max_tokens -> length), never an empty id, and `model` echoes the id the client requested — the model that actually answered stays in cortiq.selected_model and X-Cortiq-Selected-Model. Streaming is re-normalised rather than forwarded verbatim: a leading role delta, message-shaped chunks converted to delta, one terminal finish_reason, always a trailing data: [DONE], and an SSE error event when the upstream dies mid-flight. Adds X-Accel-Buffering: no so a fronting proxy cannot buffer the stream into a hang. GET /v1/models now lists managed [[cmf.servers]] models with their live running/healthy state, so clients can pin a local .cmf directly. Also strips client bookkeeping keys (chat_id, session_id, metadata, features, ...) from the upstream call and drops the leftover [to_wire] debug prints.
This commit is contained in:
+42
-2
@@ -15,13 +15,53 @@ pub enum RoutingDirective {
|
||||
Pinned { model_id: String },
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
pub struct Message {
|
||||
pub role: String, // system | user | assistant | tool
|
||||
#[serde(default)]
|
||||
/// Always a plain string internally. On the wire OpenAI clients legitimately
|
||||
/// send `null` (assistant turns that only carry tool calls) or an array of
|
||||
/// content parts (Open WebUI and every other multimodal UI), so
|
||||
/// [`de_content`] flattens both shapes instead of rejecting the request.
|
||||
#[serde(default, deserialize_with = "de_content")]
|
||||
pub content: String,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub tool_calls: Vec<serde_json::Value>,
|
||||
/// Speaker name (`messages[].name`) — proxied through when the client sends it.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name: Option<String>,
|
||||
/// Set on `role = "tool"` replies; dropping it breaks multi-turn tool calling.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub tool_call_id: Option<String>,
|
||||
}
|
||||
|
||||
/// Deserialize `messages[].content` from any shape an OpenAI-compatible client
|
||||
/// may send: a string, `null`, or an array of content parts. Text parts are
|
||||
/// concatenated; non-text parts (images, audio, files) are skipped — this
|
||||
/// gateway routes text, and a picture must not turn into a 422.
|
||||
fn de_content<'de, D>(d: D) -> std::result::Result<String, D::Error>
|
||||
where
|
||||
D: serde::Deserializer<'de>,
|
||||
{
|
||||
use serde::Deserialize as _;
|
||||
Ok(match serde_json::Value::deserialize(d)? {
|
||||
serde_json::Value::Null => String::new(),
|
||||
serde_json::Value::String(s) => s,
|
||||
serde_json::Value::Array(parts) => {
|
||||
let mut out = String::new();
|
||||
for part in parts {
|
||||
// {"type":"text","text":"..."} / {"type":"input_text","text":"..."}
|
||||
let text = part
|
||||
.get("text")
|
||||
.and_then(|t| t.as_str())
|
||||
.or_else(|| part.as_str());
|
||||
if let Some(t) = text {
|
||||
out.push_str(t);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
other => other.to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default)]
|
||||
|
||||
Reference in New Issue
Block a user