Repository navigation
fix(controller): stop sending a second response after a failed stream - #615
Merged
shipiyouniao merged 1 commit intoNov 7, 2026
Merged
shipiyouniao merged 1 commit into
shipiyouniao merged 1 commit into
Conversation
Once StreamResponse.prepare() has written the status line and headers, the catch-all handler in frontend.py still returned web.Response(status=500). aiohttp appended that response to the unfinished chunked body, so clients received two status lines in one response: C-parser clients hang until their own timeout and Python-parser clients report invalid chunk framing. Treat prepare() as the commit point. After it, _relay_stream_response() logs and aborts the connection instead of starting a new response; before it, the existing JSON 500 path is unchanged. Both completion handlers now share one proxy helper, and result.release() still runs in finally so upstream cleanup and backend accounting are untouched. Add tests/test_frontend_streaming.py, which reads the raw downstream bytes and asserts a single status line, the healthy chunked terminator, and the pre-header error path. Fixes ovg-project#610
shipiyouniao
approved these changes
Nov 7, 2026
shipiyouniao
left a comment
Collaborator
There was a problem hiding this comment.
Verified both completion endpoints through the normal frontend CLI and real router against a controlled HTTP backend (Python 3.11.15, aiohttp 3.14.3).
Base e0e4059 emits a second status line on upstream truncation. This head emits one, preserves the first SSE event, and closes without a final chunk, including keep-alive requests. Healthy streaming, non-streaming JSON, missing-model and routing-error paths also passed.
The new tests fail 2/6 on base and pass 6/6 on head. All nine CI checks are green. No blocking findings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes #610.
The controller's completion proxy answered with a fresh
web.Response(status=500)from its catch-all
excepteven afterStreamResponse.prepare()had already put thestatus line and headers on the wire. aiohttp then appended a second status line and
headers to the unfinished chunked body. With aiohttp 3.13/3.14, C-parser clients hang
until their own timeout and Python-parser clients report invalid chunk framing.
Changes
controller/frontend.pyhandle_completion()/handle_chat_completion()now share_proxy_completion_request(); the duplicated ~55-line bodies are gone and eachhandler keeps its own upstream path and log label.
_relay_stream_response()commits the response only afterprepare()succeeds.A failure past that point logs, truncates the connection
(
_abort_downstream_stream()), and returns the already-started response, so nosecond HTTP response is ever written. A failure before
prepare()stillpropagates and becomes the ordinary JSON 500, as before.
result.release()still runs infinallyon every path, so upstream connectioncleanup and router/backend accounting are unchanged.
web.StreamResponse, which is what thestreaming path actually returns.
Tests
New
tests/test_frontend_streaming.py, registered intests/manifests/cpu.txt(only needs aiohttp, which the CPU workflow already installs). It runs the real
handlers over a real aiohttp socket and reads the raw response bytes with a
hand-written HTTP request:
HTTP/1.1status line, no500, first SSE event still delivered, body left truncated (parametrised overboth endpoints);
0\r\n\r\n;model-> 400 and the router is never called.Against the unfixed
frontend.pythe two truncation tests fail withassert 2 == data.count(b"HTTP/1.1 "); with the fix all six pass.Verification