You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Replace the GitHub proxy with git over CORS proxy #1467
Let's provide a scalable way for Playground to interact with remote resources and GitHub repositories. The current one, github-proxy.com, might be challenging to scale in the longer term.
Long term, we might run into issues with the actual throughput. If and when that happens, let’s explore edge-caching zip files, queuing and rate limiting requests, size limits for cloned repos and artifacts, maybe keeping hot clones of 10 most popular repos etc.
Let's move all the Git-related computations (fetching, computing deltas, decompression etc.) into the browser – no server, no problem (with CPUs, memory, or storage). PHP and JavaScript can talk to Git directly. We'll still need a CORS proxy to make the repositories accessible via fetch(), but that's much easier to scale and maintain.
A CORS proxy – we'll need to host a streaming CORS proxy on a separate domain, like cors.playground.wordpress.net or even cors-playground.wordpress.net or playground-cors.com to avoid any subdomains-related risks.
I think we could have the same for SVN. We’d need to implement svn+ssh over a CORS proxy which would take some time so I’m not prioritizing it, but it’s a nice future option to have.
Also porting that to PHP would unlock installing WordPress plugins from the latest trunk version and updating them on new commit. Probably not good for production sites, but I can think of a few use-cases already.
"Git operations do not consume part of your API rate limit, as there are no API calls made to GitHub.com via the git client.
We don't have any hard limits for clones, though we may delay requests if they come in fast enough to potentially cause overload on one of our servers (this would be determined by the amount of load being placed on our servers at the time of the clones, and would have to exceptionally high to occur). So while Git operations do have dynamic limits, this might result in slower clones, but shouldn't cause any failures.
However, they do on REST API requests:
In terms of API requests: unauthenticated ones are typically limited to ~5000/hr per IP address, 5000/hr per authenticated user, and significantly lower for anonymous requests (typically ~100/hr or less)."
@adamziel, I'm not sure how to answer because there are multiple kinds of things mentioned in this issue, including both the GitHub proxy and plugin-proxy.php.
It seems like the issue title "Replace the GitHub proxy with git over CORS proxy" could be considered complete because of GitDirectoryResource using cors-proxy.php automatically, but we still have plugin-proxy.php references like these for WP core and Gutenberg PR previews:
Activity
[-]Scaling GotHub proxy[/-][+]Scaling GitHub proxy[/+]adamziel commentedon Jun 21, 2024
Let's move all the Git-related computations (fetching, computing deltas, decompression etc.) into the browser – no server, no problem (with CPUs, memory, or storage). PHP and JavaScript can talk to Git directly. We'll still need a CORS proxy to make the repositories accessible via
fetch(), but that's much easier to scale and maintain.For v1 we need:
cors.playground.wordpress.netor evencors-playground.wordpress.netorplayground-cors.comto avoid any subdomains-related risks.adamziel commentedon Jun 21, 2024
I think we could have the same for SVN. We’d need to implement svn+ssh over a CORS proxy which would take some time so I’m not prioritizing it, but it’s a nice future option to have.
Also porting that to PHP would unlock installing WordPress plugins from the latest trunk version and updating them on new commit. Probably not good for production sites, but I can think of a few use-cases already.
[-]Scaling GitHub proxy[/-][+]Replace GitHub proxy with git over CORS proxy[/+]Explore: PHP CORS Proxy
PHP CORS Proxy (#1546)
adamziel commentedon Aug 2, 2024
Here's the way forward for this issue:
playground-proxy.wordpress.net.plugin-proxy.phpendpoint in favor of that CORS proxy.git clonevia JavaScript(), do a sparse checkout as needed.gitresource type for referencing git repositories.gitresource type.Optionally, we could migrate the PHP CORS proxy to Async\HttpClient to give that library more real-world testing.
GitHub does not have specific rate-limiting for the regular git client operations:
However, they do on REST API requests:
[-]Replace GitHub proxy with git over CORS proxy[/-][+]Replace the GitHub proxy with git over CORS proxy[/+]adamziel commentedon Oct 23, 2024
@brandonpayton can we resolve this one now?
brandonpayton commentedon Oct 31, 2024
@adamziel, I'm not sure how to answer because there are multiple kinds of things mentioned in this issue, including both the GitHub proxy and plugin-proxy.php.
It seems like the issue title "Replace the GitHub proxy with git over CORS proxy" could be considered complete because of GitDirectoryResource using cors-proxy.php automatically, but we still have plugin-proxy.php references like these for WP core and Gutenberg PR previews:
In case it helps close this issue, I created #1970 to track migration away from plugin-proxy.php.
There are also other CORS proxy changes to make in response to feedback from A8C Systems, including switching to a separate, dedicated domain.
brandonpayton commentedon Nov 1, 2024
Let's wait to close this until we're relying entirely on the CORS proxy.
brandonpayton commentedon Nov 1, 2024
Actually, I forgot that we created #1970 "Replace uses of plugin-proxy.php with generic CORS proxy" for that.
I'll go ahead and close this.