Repository navigation
[Cors Proxy] Support Transfer-Encoding: Chunked - #2077
Merged
Merged
Conversation
Adjusts the CORS Proxy to relay chunked responses using chunks. ## Implementation Some APIs use fixed content-length and respond with a predictable response body. Other APIs respond with chunks. Before this PR, whenever the CORS proxy received a chunked response, it would only relay the `Transfer-Encoding: Chunked` header without actually transmitting the response body as chunks. Chunks are an essential HTTP 1.1 feature that cannot be disabled, We also can't just filter out that header in the proxy response as the client expects either a Content-Length or body chunks, and we may not know the Content-length upfront. With this PR, the CORS proxy outputs its response using the chunked encoding whenever the remote endpoint streams its response using the chunked encoding. ## Testing * Run `nx start playground-php-cors-proxy` * Fixed-length response: Go to http://127.0.0.1:5263/cors-proxy.php?https://raw.githubusercontent.com/WordPress/wordpress-playground/5e5ba3e0f5b984ceadd5cbe6e661828c14621d25/README.md * Confirm it loaded the file * Chunked encoding: Go to http://127.0.0.1:5263/cors-proxy.php?https://api.wordpress.org/plugins/update-check/1.1/ * Confirm the response says the word `error` * Error handling: Go to http://127.0.0.1:5263/cors-proxy.php?https://nosuchsite.playground.wordpress.net * Confirm the response says Bad Request Hostname could not be resolved
adamziel
commented
Dec 12, 2024
| header('Access-Control-Allow-Credentials: true'); | ||
| header('Access-Control-Allow-Methods: GET, POST, OPTIONS'); | ||
| header('Access-Control-Allow-Headers: Accept, Authorization, Content-Type, git-protocol'); | ||
| header('Access-Control-Allow-Headers: Accept, Authorization, Content-Type, git-protocol, wp_blog, wp_install'); |
Collaborator
Author
There was a problem hiding this comment.
Casual addition – these two headers are used by WordPress core when requesting api.w.org
edisplay
pushed a commit
to edisplay/wordpress-playground
that referenced
this pull request
Dec 17, 2024
…he latest release (WordPress#2056) Fixes a regression created in WordPress#2077. The minified nightly build accidentally became a minified build of the latest WordPress version. In the multitude of resolveWordPressRelease() calls used in building the minified WordPress releases, the last call in the chain started transforming the nightly URL into a 6.7.1 release URL. This PR removes that one final call, and also simplifies the Playground CLI build pipeline. ## Testing instructions Apply this PR, switch WP version to nightly, confirm in wp-=admin it's indeed the nightly release
adamziel
added a commit
that referenced
this pull request
Jan 2, 2025
Adds support for `transfer-encoding: chunked` on Apache, Nginx, and other web servers. #2077 added support for `transfer-encoding: chunked` in a way that works on a local PHP dev server. The proxy manually chunks the output bytes and outputs each chunk's header, separator, body, and trailer. However, it doesn't work on playground.wordpress.net because the web server there handles the chunked encoding on its own. The bytes echoed by cors-proxy.php are treated as body bytes, which messes up the response body. This PR restricts the manual chunking to a local CLI dev server. ## Testing instructions Run cors-proxy.php in Apache or so and confirm that requesting resources served with chunked encoding works with this patch but not without it. One such URL is https://adamadam.blog/feed/.
adamziel
added a commit
that referenced
this pull request
Jan 9, 2025
#2114) Adds support for `transfer-encoding: chunked` on Apache, Nginx, and other web servers. #2077 added support for `transfer-encoding: chunked` in a way that works on a local PHP dev server. The proxy manually chunks the output bytes and outputs each chunk's header, separator, body, and trailer. However, it doesn't work on playground.wordpress.net because the web server there handles the chunked encoding on its own. The bytes echoed by cors-proxy.php are treated as body bytes, which messes up the response body. This PR restricts the manual chunking to a local CLI dev server. ## Testing instructions Run cors-proxy.php in Apache or so and confirm that requesting resources served with chunked encoding works with this patch but not without it. One such URL is https://adamadam.blog/feed/.
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 27, 2026
The proxy stopped relaying Content-Length in #2077, which restructured the header callback to support chunked responses. Relay it again for fixed-length responses, deciding after all headers arrive because HTTP ignores Content-Length when Transfer-Encoding: chunked is present. With Content-Length relayed, clients detect responses cut short by a target failure, and HEAD becomes useful for learning a file's size before requesting ranges. Accept HEAD, skip the response size cap for it since no body is relayed, and tell curl not to wait for a body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brandonpayton
added a commit
that referenced
this pull request
Sep 28, 2026
…sponses (#4365) ## Motivation for the change, related issues The CORS proxy has not relayed the target's `Content-Length` since #2077. That PR restructured the response header callback to support chunked responses, and every special-cased header gained an early return. #2077 does not discuss dropping `Content-Length`, and the proxy relayed it before that. Without it: - Clients can't detect a response cut short by a target failure. The body just ends early. - Full responses carry no size. Playground's blueprint download progress falls back to a 5MB estimate, for example. - `HEAD` would be useless even if allowed. The proxy rejects it with 405 today. `HEAD` plus `Content-Length` is the usual way for a client to learn a file's size before reading ranges, e.g. to locate a ZIP archive's central directory. This PR builds on #4362, which added range request support and runs the e2e tests from `test.sh`, and #4364, which stops relaying server-control headers. Both have merged, and this PR is based on `trunk`. ## Implementation details - **Relay `Content-Length` for fixed-length responses.** The header callback holds the value and emits it when the header block ends, but only if the response has no `Transfer-Encoding`. RFC 9112 §6.3 says `Transfer-Encoding` overrides `Content-Length`, and headers can arrive in either order. The held value resets for each header block, e.g. after a `100 Continue`, and only digit-only values are relayed. - Chunked responses are unchanged and still relayed as chunks. - **Support `HEAD`.** Accept it in the method check, list it in `Allow` and `Access-Control-Allow-Methods`, and set `CURLOPT_NOBODY` so curl doesn't wait for the announced body. `HEAD` is a CORS-safelisted method, and `Content-Length` is a safelisted response header, so browsers need no preflight or exposure for this. - `HEAD` also asks the target for `Accept-Encoding: identity`, as ranged requests do. Clients use the `HEAD` size to compute range offsets, so it must describe the same unencoded bytes the ranges address. - The response size cap limits relayed bodies, so it no longer applies to `HEAD`. A `HEAD` on a multi-GiB file returns its size instead of `413`. - When the proxy replaces a response with its own `413` or `502`, the relayed `Content-Length` is removed with the rest of the target's headers, and the held value is cleared. - **Fix interim 1xx responses (existing bug).** The proxy took its status from the first status line curl reported and relayed every header block. A target that sent `103 Early Hints` or `100 Continue` before its final response made the proxy answer with that interim status and headers, followed by the final body, which clients reject. The proxy now sets its status from the final status line and skips interim header blocks entirely. A `101 Switching Protocols` isn't skipped: the proxy can't relay another protocol, so it answers with `502` instead of presenting the other protocol's bytes as a response body. - The request body size check now applies only to `POST`, the only accepted method with a body. Previously any non-`GET` method read `$_SERVER['CONTENT_LENGTH']`. **Effects inside Playground:** PHP.wasm's outbound HTTP (`tcp-over-fetch-websocket.ts`) strips `Content-Length` before handing responses to PHP, so PHP sees no difference there. `HEAD` requests from PHP that go through the proxy, e.g. `wp_remote_head()`, now succeed instead of getting `405`. When the client receives the target's `Content-Length`, a proxied body cut short now makes `fetch()` fail instead of returning short data. That doesn't hold for chunked responses, or when a server in front of PHP compresses the response and drops `Content-Length`; the README says so. Blueprint resource downloads and URL-based file reading gain real sizes for progress. When a target compresses a response, `Content-Length` is the compressed size, so a download step's progress can show complete before the download ends. That's cosmetic, since the overall operation can't finish early, and it already happens with the 5MB fallback. ## Testing Instructions (or ideally a Blueprint) ```bash cd packages/playground/php-cors-proxy bash test.sh ``` Expect PHPUnit to pass, followed by `All 66 assertions passed.` from the e2e script. New and updated e2e tests: - A fixed-length response relays `Content-Length`. - A chunked response relays `Transfer-Encoding: chunked` but not `Content-Length`, even when the target sends `Content-Length` before `Transfer-Encoding: chunked`. A small raw-socket server, `tests/e2e/raw-upstream-server.php`, produces that response, because PHP's built-in server can't send chunked responses. I confirmed this test fails if `Content-Length` is relayed as soon as it arrives, or without the chunked check. - `HEAD` on a 4 GiB file returns `200` with `Content-Length: 4294967296`, and `HEAD` asks the target for `Accept-Encoding: identity`. - A target that sends `103 Early Hints` before its `200` produces a `200` response with the final body and without the interim `Link` header. The raw server also produces this response, and a `101 Switching Protocols` followed by other bytes, which gets a `502` with only the proxy's headers. - A target that fails mid-body now gives the client an incomplete-response error (`CURLE_PARTIAL_FILE`) instead of a complete-looking short body. The test also checks that the partial body holds only the target's bytes; the test helper now captures bodies through a callback so failed transfers keep what arrived. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * The CORS proxy supports `HEAD` requests, returning response headers without a body and requesting identity encoding. * Target `Content-Length` is relayed only when the response is not chunked. Interim response headers are skipped, and partial responses can be detected using the announced length or, for range responses, `Content-Range`. * Protocol-switching responses are rejected with a 502 error. Response-size limits do not apply to `HEAD` requests. * **Documentation** * Clarified supported methods and guidance for detecting truncated responses. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adjusts the CORS Proxy to relay chunked responses using chunks.
Implementation
Some APIs use fixed content-length and respond with a predictable response body. Other APIs respond with chunks.
Before this PR, whenever the CORS proxy received a chunked response, it would only relay the
Transfer-Encoding: Chunkedheader without actually transmitting the response body as chunks.Chunks are an essential HTTP 1.1 feature that cannot be disabled, We also can't just filter out that header in the proxy response as the client expects either a Content-Length or body chunks, and we may not know the Content-length upfront.
With this PR, the CORS proxy outputs its response using the chunked encoding whenever the remote endpoint streams its response using the chunked encoding.
Testing
nx start playground-php-cors-proxyerror