Resolve OpenSSL 4.0 build issues - #13482
Closed
cmcfarlen wants to merge 2 commits into
Closed
Conversation
Fedora 45 ships OpenSSL 4.0, which constifies several X509/ASN1 accessors and makes ASN1_STRING opaque. Hardcoding const breaks OpenSSL 1.1.1, where X509_NAME_get_index_by_NID() still takes a non-const X509_NAME, so deduce the pointer and function-pointer types from the accessors instead. That keeps a single source form building on 1.1.1, 3.x and 4.0 with no version macros. Co-authored-by: Jered Floyd <jered@redhat.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates multiple ATS core components, plugins, examples, and tests to build cleanly across OpenSSL 1.1.1, 3.x, and 4.0 by avoiding direct access to newly-opaque ASN1 internals and by letting constness/function-pointer types be deduced from OpenSSL accessors (instead of hardcoding types that differ across versions).
Changes:
- Replace direct
ASN1_STRINGfield access (->data,->length,->type) with accessor APIs (ASN1_STRING_get0_data,ASN1_STRING_length,ASN1_STRING_type). - Update X509/X509_NAME usage to be const-correct across OpenSSL versions via
auto */decltype(...)-based deduction. - Adjust certificate-subject mutation patterns (dup + set back) where OpenSSL now exposes subject names as const.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/tools/plugins/ssl_client_verify_test.cc | Deduces subject name constness and updates CN extraction to use ASN1 accessors. |
| src/tscore/X509HostnameValidator.cc | Switches hostname validation string checks to ASN1 accessor APIs and deduced subject name constness. |
| src/iocore/net/unit_tests/test_SSLDHParams.cc | Updates self-signed cert generation to dup/replace subject name for OpenSSL 4 constness changes. |
| src/iocore/net/SSLUtils.cc | Makes ASN1 string duplication const-correct and deduces subject name constness for CN extraction. |
| src/iocore/net/SSLNetVConnection.cc | Adjusts debug helper signature to accept const X509_NAME *. |
| src/iocore/net/OCSPStapling.cc | Replaces direct ASN1 buffer access with accessor APIs. |
| src/cripts/Certs.cc | Updates certificate field extraction to use ASN1 accessors and deduced getter types. |
| include/cripts/Certs.hpp | Defines deduced getter typedefs (decltype(&X509_get_...)) to keep function-pointer types portable across OpenSSL versions. |
| src/api/InkAPI.cc | Updates CN extraction types and ASN1 access for OpenSSL 4 compatibility. |
| plugins/lua/ts_lua_client_cert_helpers.h | Updates X509 name helper constness and uses ASN1 accessors for signature formatting. |
| plugins/experimental/txn_box/plugin/src/ts_util.cc | Uses decltype(X509_get_subject_name(nullptr)) to deduce X509_NAME* constness for nid lookup. |
| plugins/experimental/sslheaders/expand.cc | Makes issuer/subject name pointers const and uses ASN1 accessors for signature dump. |
| plugins/experimental/cert_reporting_tool/cert_reporting_tool.cc | Makes subject name pointer const to match OpenSSL accessor changes. |
| plugins/certifier/certifier.cc | Dups subject name before mutation and introduces an RAII type for X509_NAME. |
| example/plugins/c-api/client_context_dump/client_context_dump.cc | Deduces subject name constness for compatibility. |
Drop the unnecessary const cast when duplicating an ASN1 string, assert the subject CN is actually added in the DH params test, build the CN string directly from the ASN1 buffer instead of strndup, and use an explicit deleter for the X509_NAME unique_ptr rather than specializing std::default_delete.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
Suppressed comments (1)
tests/tools/plugins/ssl_client_verify_test.cc:78
- The SAN (subjectAltName) GEN_DNS path still uses strndup()/free() and then passes the resulting char* to check_name(const std::string&). If strndup() returns nullptr (allocation failure), constructing a std::string from it is UB and can crash. Since you already have pointer+length from OpenSSL, you can avoid the allocation entirely by constructing a std::string directly (same approach as the CN path above).
std::string subj_name{reinterpret_cast<const char *>(ASN1_STRING_get0_data(cn)), static_cast<size_t>(ASN1_STRING_length(cn))};
retval = check_name(subj_name);
}
}
if (!retval) {
3 tasks
maskit
added a commit
to maskit/trafficserver
that referenced
this pull request
Aug 3, 2026
certifier.cc freed the duplicated X509_NAME manually on every path; switch to a scoped_X509_NAME unique_ptr matching the file existing scoped_X509/scoped_EVP_PKEY/scoped_SSL_CTX aliases so no path can forget to free it. ts_util.cc templated ssl_value_for just to defer the parameter type to the caller; decltype(X509_get_subject_name(nullptr)) deduces the same pointer type directly without turning it into a template. Also drop the last C-style cast this series introduced in X509HostnameValidator.cc in favor of reinterpret_cast.
maskit
added a commit
that referenced
this pull request
Aug 5, 2026
* Resolve OpenSSL 4.0 build issues - accessors for ASN1_STRING and const-iness * Ran clang-format to fix formatting * Fix X509_NAME_get_index_by_NID const mismatch on OpenSSL 1.1.1 OpenSSL 1.1.1 declares the first argument as non-const while newer releases declare it const, so building against different versions failed depending on which signature was in effect. Add a const_cast at each call site to keep the const-qualified variables introduced by this PR buildable against both, matching the existing precedent in OCSPStapling.cc. * Drop const_cast in favor of auto for local X509_NAME variables X509_get_subject_name and X509_NAME_get_index_by_NID have their argument constness changed together across OpenSSL versions, so a local variable declared with auto tracks whatever type is correct for the OpenSSL version in use, without a cast. This applies only to the two purely local variables; ts_util.cc ssl_value_for keeps its const_cast since its parameter type is shared across multiple callers. * Fix const-cast style and X509_NAME leak flagged by Copilot review X509HostnameValidator.cc cast ASN1_STRING_get0_data return value to non-const before an ats_strndup call that only wants const char *; drop the const instead of adding it back needlessly. certifier.cc leaked the duplicated X509_NAME on the X509_NAME_add_entry_by_txt failure path, and never checked X509_NAME_dup for allocation failure. * Make Cripts X509 accessors OpenSSL-4-compatible CertBase::X509Value took hardcoded function pointer types for X509_get_subject_name, X509_get_issuer_name, X509_getm_notBefore, and X509_getm_notAfter, but those accessors change constness in different directions across OpenSSL versions, so no single hardcoded signature builds everywhere. Deduce the parameter type from the actual accessor via decltype instead. Signature::_load and _write_ip_address also read ASN1_STRING fields directly, which breaks once the struct is opaque; switch to the accessor functions used elsewhere in this codebase. * Fix lua plugin X509_NAME/ASN1_STRING OpenSSL 4 compatibility get_x509_name_string only reads through the name via X509_NAME_print_ex, so accept a const X509_NAME * to match callers that pass X509_get_subject_name/X509_get_issuer_name results directly. get_x509_signature_string read the ASN1_STRING struct fields directly, which breaks once the struct is opaque; use ASN1_STRING_get0_data/ASN1_STRING_length instead. * Fix self-signed test cert losing its CN under OpenSSL 4 make_cert_and_key mutated the X509_NAME returned by X509_get_subject_name in place, which stops compiling once that accessor can return const, and was already fragile since the returned name is only a view into the certificate internal state. Duplicate it, add the CN to the duplicate, and set it back as both subject and issuer name since this is a self-signed certificate. * Fix OpenSSL 4 X509_NAME const mismatches in example plugin and test tool client_context_dump.cc, verify_cert.cc, and ssl_client_verify_test.cc all held X509_get_subject_name/X509_NAME_get_entry/ X509_NAME_ENTRY_get_data results in hardcoded non-const locals or parameters, which stops compiling once those accessors return const. Switch to auto for the local variables and const for the debug_certificate parameter, matching the read-only usage in each case. * Drop last const_cast in txn_box ssl_value_for via a template ssl_value_for is shared by four callers, each already deducing its X509_NAME pointer type with auto from X509_get_subject_name or X509_get_issuer_name, so unlike the other three call sites fixed earlier in this series, a single hardcoded parameter type cannot track the underlying accessor across OpenSSL versions. Templating the parameter on the callers deduced type removes the cast entirely. * Alias the Cripts X509 getter decltypes for readability Bare decltype(&X509_get_subject_name) in a parameter list reads poorly at each of the four call sites; name each getter type once via using so the declarations and out-of-line definitions just say what kind of accessor they take. * Adopt RAII and decltype(func(nullptr)) patterns from PR #13482 certifier.cc freed the duplicated X509_NAME manually on every path; switch to a scoped_X509_NAME unique_ptr matching the file existing scoped_X509/scoped_EVP_PKEY/scoped_SSL_CTX aliases so no path can forget to free it. ts_util.cc templated ssl_value_for just to defer the parameter type to the caller; decltype(X509_get_subject_name(nullptr)) deduces the same pointer type directly without turning it into a template. Also drop the last C-style cast this series introduced in X509HostnameValidator.cc in favor of reinterpret_cast. --------- Co-authored-by: Jered Floyd <jered@redhat.com>
cmcfarlen
pushed a commit
that referenced
this pull request
Aug 6, 2026
* Resolve OpenSSL 4.0 build issues - accessors for ASN1_STRING and const-iness * Ran clang-format to fix formatting * Fix X509_NAME_get_index_by_NID const mismatch on OpenSSL 1.1.1 OpenSSL 1.1.1 declares the first argument as non-const while newer releases declare it const, so building against different versions failed depending on which signature was in effect. Add a const_cast at each call site to keep the const-qualified variables introduced by this PR buildable against both, matching the existing precedent in OCSPStapling.cc. * Drop const_cast in favor of auto for local X509_NAME variables X509_get_subject_name and X509_NAME_get_index_by_NID have their argument constness changed together across OpenSSL versions, so a local variable declared with auto tracks whatever type is correct for the OpenSSL version in use, without a cast. This applies only to the two purely local variables; ts_util.cc ssl_value_for keeps its const_cast since its parameter type is shared across multiple callers. * Fix const-cast style and X509_NAME leak flagged by Copilot review X509HostnameValidator.cc cast ASN1_STRING_get0_data return value to non-const before an ats_strndup call that only wants const char *; drop the const instead of adding it back needlessly. certifier.cc leaked the duplicated X509_NAME on the X509_NAME_add_entry_by_txt failure path, and never checked X509_NAME_dup for allocation failure. * Make Cripts X509 accessors OpenSSL-4-compatible CertBase::X509Value took hardcoded function pointer types for X509_get_subject_name, X509_get_issuer_name, X509_getm_notBefore, and X509_getm_notAfter, but those accessors change constness in different directions across OpenSSL versions, so no single hardcoded signature builds everywhere. Deduce the parameter type from the actual accessor via decltype instead. Signature::_load and _write_ip_address also read ASN1_STRING fields directly, which breaks once the struct is opaque; switch to the accessor functions used elsewhere in this codebase. * Fix lua plugin X509_NAME/ASN1_STRING OpenSSL 4 compatibility get_x509_name_string only reads through the name via X509_NAME_print_ex, so accept a const X509_NAME * to match callers that pass X509_get_subject_name/X509_get_issuer_name results directly. get_x509_signature_string read the ASN1_STRING struct fields directly, which breaks once the struct is opaque; use ASN1_STRING_get0_data/ASN1_STRING_length instead. * Fix self-signed test cert losing its CN under OpenSSL 4 make_cert_and_key mutated the X509_NAME returned by X509_get_subject_name in place, which stops compiling once that accessor can return const, and was already fragile since the returned name is only a view into the certificate internal state. Duplicate it, add the CN to the duplicate, and set it back as both subject and issuer name since this is a self-signed certificate. * Fix OpenSSL 4 X509_NAME const mismatches in example plugin and test tool client_context_dump.cc, verify_cert.cc, and ssl_client_verify_test.cc all held X509_get_subject_name/X509_NAME_get_entry/ X509_NAME_ENTRY_get_data results in hardcoded non-const locals or parameters, which stops compiling once those accessors return const. Switch to auto for the local variables and const for the debug_certificate parameter, matching the read-only usage in each case. * Drop last const_cast in txn_box ssl_value_for via a template ssl_value_for is shared by four callers, each already deducing its X509_NAME pointer type with auto from X509_get_subject_name or X509_get_issuer_name, so unlike the other three call sites fixed earlier in this series, a single hardcoded parameter type cannot track the underlying accessor across OpenSSL versions. Templating the parameter on the callers deduced type removes the cast entirely. * Alias the Cripts X509 getter decltypes for readability Bare decltype(&X509_get_subject_name) in a parameter list reads poorly at each of the four call sites; name each getter type once via using so the declarations and out-of-line definitions just say what kind of accessor they take. * Adopt RAII and decltype(func(nullptr)) patterns from PR #13482 certifier.cc freed the duplicated X509_NAME manually on every path; switch to a scoped_X509_NAME unique_ptr matching the file existing scoped_X509/scoped_EVP_PKEY/scoped_SSL_CTX aliases so no path can forget to free it. ts_util.cc templated ssl_value_for just to defer the parameter type to the caller; decltype(X509_get_subject_name(nullptr)) deduces the same pointer type directly without turning it into a template. Also drop the last C-style cast this series introduced in X509HostnameValidator.cc in favor of reinterpret_cast. --------- Co-authored-by: Jered Floyd <jered@redhat.com> (cherry picked from commit bdefaf9)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fedora 45 ships OpenSSL 4.0, which breaks the ATS build. This supersedes #13440 by
@jeredfloyd, which had the right diagnosis but did not build on OpenSSL 1.1.1 and
missed several affected files. Jered is credited via
Co-authored-by.Resolves #13427.
Root cause
Diffing the actual headers from OpenSSL 1.1.1f and 4.0.1, exactly one API changes in
a way that can break compilation:
X509_NAME_get_index_by_NIDX509_NAME *const X509_NAME *Others changed only their return type to const in 4.0 (
X509_get_subject_name,X509_get_issuer_name,X509_NAME_get_entry,X509_NAME_ENTRY_get_data,X509_get0_pubkey_bitstr), andASN1_STRINGbecame opaque. Receiving those into aconst-qualified variable is legal on every version.
So hardcoding
constis required on OpenSSL 4 but illegal on 1.1.1 — which is why#13440 failed the Ubuntu 20.04 and FreeBSD 13.1 jobs.
Approach
Let the type be deduced from the accessor rather than hardcoded —
auto *forlocals,
decltype(&X509_get_subject_name)for parameter and function-pointer types.One source form compiles on 1.1.1, 3.x and 4.0 with no version macros and no
const_cast.One case makes this mandatory rather than merely tidy:
X509_getm_notBefore/X509_getm_notAfterchanged const in the opposite direction (they tookconst X509 *in 1.1.1 and takeX509 *in 4.0), so no hardcoded spelling workseverywhere.
Beyond #13440
These are also broken on OpenSSL 4 and were not covered by #13440:
plugins/lua/ts_lua_client_cert_helpers.h,plugins/lua/ts_lua_client_request.ccsrc/cripts/Certs.cc,include/cripts/Certs.hpp— same opaque-ASN1_STRINGissue that Resolve OpenSSL 4.0 build issues #13440 fixed in sslheaders, plus two function-pointer types
src/iocore/net/unit_tests/test_SSLDHParams.ccexample/plugins/c-api/client_context_dump/client_context_dump.cctests/tools/plugins/ssl_client_verify_test.ccTwo fixes here are not purely mechanical:
plugins/certifier/certifier.cc— theX509_NAME_dupapproach needs the copyfreed on the
X509_NAME_add_entry_by_txtfailure path and thedupresultnull-checked. Reworked onto the file's existing
std::unique_ptridiom.test_SSLDHParams.cc— this relied on mutating the subject name in place.Once the accessor returns const you must dup and then call both
X509_set_subject_nameandX509_set_issuer_name, or the generated self-signedcert silently loses its CN.
Verification
Full builds with
BUILD_EXPERIMENTAL_PLUGINS=ONandBUILD_REGRESSION_TESTING=ON,Cripts enabled where the platform's fmt allows it, plus
test_tscoreandtest_net:All on linux/arm64.
Caveats:
test_SSLDHParams.ccis gated behindSSLLIB_IS_AT_LEAST_OPENSSL3, so that changeis only exercised on 3.x/4.0.
minimum — so the Cripts changes are verified on OpenSSL 3.5 and 4.0 only.
src/tscore/unit_tests/test_X509HostnameValidator.ccexists but is not wired intoany CMake target, so
validate_hostnamehas no unit coverage. Pre-existing; leftalone here.