Repository navigation
client: reuse one no-redirect opener for calls without an SSL context (#722) - #807
Merged
LoopGlitch26 merged 1 commit intoNov 13, 2026
Merged
Conversation
…Cygnux-Labs#722) no_redirect_opener() built a new opener per call, and building one makes an SSL context (Python 3.12 on), which on Windows loads the system's certificates each time: ~40 ms a call for every webhook, witness, Slack, Rekor, OTLP, reviewer and remote-ingest request. A call without a context now gets the same opener; a caller's own context still gets its own. The ProxyHandler({}) and the no-redirect handler are unchanged, and neither the OpenerDirector nor its handlers keep per-request state, so the opener is shared across threads. Tests: two calls share one opener and make at most one SSL context, and a redirect is still refused on the second call, through the shared opener. Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
Member
|
Read it. This is what we agreed:
|
Member
|
Merged, thank you. One shared opener for calls without an SSL context removes the per-call setup cost you measured on Windows, and both of your #722 fixes are now on main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part (3) of #722, step (2), as agreed there: one opener for every call that brings no SSL context.
Why
no_redirect_opener()built a new opener on every call. On Python 3.12 and later,urllib.request.HTTPSHandler()creates its SSL context when it's built, andbuild_openeradds one even forhttp://URLs. On Windows, that loads the system certificate store each time. In the e20 test's profile, it cost ~57 s across ~1,100 calls, about 40 ms a call. The webhook, witness, Slack, Rekor, OTLP, reviewer and remote-ingest senders all paid it per request.Change (
tracekit/client.py)no_redirect_opener(context=None)returns_shared_opener(), built once (functools.lru_cache(maxsize=1)). A caller that passes its owncontext(executor_tools) still gets a new opener of its own._no_redirect_opener):ProxyHandler({}), the same no-redirect handler, and the TLS handler only for a given context.Thread safety, what I checked (CPython 3.14's
urllib.requestsource; the CI runs above cover 3.9 and 3.13):OpenerDirector.open,_openand_call_chaindon't assign toself. The handler tables are filled only byadd_handler, whichbuild_openercalls once.HTTPRedirectHandler.http_error_302keeps its loop state on the request (req.redirect_dict), and ours returnsNonebefore that.AbstractHTTPHandler.do_openmakes a new connection per request.ProxyHandleronly readsself.proxies.HTTPSHandlerholds its_context, and anSSLContextis safe to share across threads. The one lazily filled cache,FileHandler.names, is class-level already, so sharing changes nothing there.Tests (
tests/test_ingest.py)test_calls_without_a_context_share_one_opener_and_its_ssl_context: two calls return the same opener and create at most one SSL context (one on 3.12+, none before, where each HTTPS connection makes its own); a caller's own context gets a different opener.test_redirect_is_not_followednow sends its request twice, so the second one goes through the shared opener. Both are refused, and the redirect target is never contacted.Checked on my fork
tests/test_ingest.py,tests/test_witness_server.pyandtests/test_review_model.pypass on windows-latest 3.13, ubuntu-latest 3.13 and ubuntu-22.04 3.9.pyflakesis clean.client.py(with a stand-in_shared_openername, so the test runs rather than failing on the import) fails on all three:AssertionError: <OpenerDirector …> is not <OpenerDirector …>. The redirect test passes on both, as it should.