Skip to content

Upgrade Protobuf to v33.6 - #29906

Merged
Tianlei Wu (tianleiwu) merged 15 commits into
mainfrom
copilot/update-protobuf-version
Aug 24, 2026
Merged

Tianlei Wu (tianleiwu) merged 15 commits into
mainfrom
copilot/update-protobuf-version

Conversation

Copilot AI commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Description

  • Native and vcpkg

    • Pin Protobuf source and prebuilt protoc archives to v33.6.
    • Update the custom vcpkg port to 6.33.6, including v33-compatible patches and tool naming.
    • Remove upstreamed compatibility patches; retain the Android linker patch.
  • Packaging and CI

    • Update Python packages to protobuf==6.33.6.
    • Update .NET packages and tooling to Google.Protobuf / Google.Protobuf.Tools 3.33.6.
    • Align Dawn's vcpkg Protobuf constraint.

Binary Size

Measured Increase

The equivalent Android minimal build changed from:

Protobuf 21.12 Protobuf 33.6 Delta
Counted ELF sections 1,450,710 1,583,982 +133,272 bytes
Percentage +9.19%
.text 1,115,548 1,215,832 +100,284
.bss 15,184 24,677 +9,493
.data 1,816 9,456 +7,640
Relocations 102,936 110,496 +7,560
.rodata 99,960 105,287 +5,327

The workflow measures the sum of all ELF sections except .relro_padding, rather than the filesystem size.

Attribution

Using identical Android NDK 28, Clang/LLD 19, MinSizeRel, ThinLTO, stripping, and section-GC settings, the linker maps attribute the source-level increase as follows:

Source Increase Share
Abseil +76,335 bytes 53%
Protobuf-lite +48,875 bytes 34%
Linker metadata and relocations +12,760 bytes 9%
ONNX Runtime +5,188 bytes 4%
Other +746 bytes <1%

The largest contributors include:

  • generated_message_tctable_lite.cc.o: about +49.5 KB
  • Abseil synchronization and mutex support: about +15.8 KB
  • Abseil floating-point formatting: about +8.4 KB
  • Abseil time-zone support: about +6.7 KB
  • Abseil Cord implementation: about +6.3 KB
  • Abseil logging and allocation support: about +8.7 KB combined

Protobuf 21.12 retained code from only three Abseil archives in this minimal binary. Protobuf 33.6 retained code from 33 Abseil archives, including synchronization, formatting, time zones, Cord, logging, allocation, stacktrace, and strings.

Both builds use libprotobuf-lite.a, not full Protobuf. The linker flags are also unchanged: ThinLTO, --gc-sections, function/data sections, -Os, and stripping are all active.

The threshold of andriod CI binary size increase is not hiding a configuration regression. Roughly 87% of the growth comes directly from:

  1. Protobuf-lite’s larger table-driven parsing runtime.
  2. Protobuf 33.6 making considerably more Abseil implementation code reachable.

The new threshold leaves only 1,170 bytes of headroom (1,583,982 measured versus 1,585,152 allowed), so it accurately reflects current size but is quite tight. Reducing the increase would require changing or patching Protobuf’s lite runtime dependencies, particularly the logging/formatting/time/synchronization chain, rather than adjusting ORT linker settings.

Motivation and Context

Newer Protobuf is required for static iOS integration so dependent libraries can share ONNX Runtime's Protobuf symbols without duplicate-symbol linker conflicts.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI changed the title [WIP] Update Protobuf to version 33.6 or greater Upgrade Protobuf to v33.6 Jul 27, 2026
@Craigacp

Copy link
Copy Markdown
Contributor

Can you bump it in Java too? Or should I make a separate PR for that?

The equivalent Java version is https://central.sonatype.com/artifact/com.google.protobuf/protobuf-java/4.33.6, and it needs updating here and here.

@tianleiwu

Copy link
Copy Markdown
Contributor

Copilot, please update protobuf-java to 4.33.6. Also use .agents/skills/ort-ci/SKILL.md to fix CI pipeline failures.

Copilot AI commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Copilot, please update protobuf-java to 4.33.6. Also use .agents/skills/ort-ci/SKILL.md to fix CI pipeline failures.

Updated all protobuf-java declarations to 4.33.6 in 39a428c. CI logs show the native failures are due to the v33.6 source archive missing from the Terrapin asset cache; the required cache onboarding cannot be performed from this PR runner.

@tianleiwu

Tianlei Wu (tianleiwu) commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

CI update:

  • The Windows Abseil C++17 configure failure is fixed in 4cfb09bdbe3aba0b4fdf05e4614ab16b7acf6fb1.
  • A local overlay install now builds abseil:x64-linux@20250814.0 and protobuf:x64-linux@6.33.6 successfully.
  • The remaining vcpkg jobs are blocked on Terrapin onboarding for the Protobuf v33.6 source archive. The self-hosted Windows run https://github.com/microsoft/onnxruntime/actions/runs/32594211937 reached Protobuf, but Terrapin reported Failed to acquire valid auth token ... for Terrapin upload request.

Could EngSys/1ES cache owner mirror this exact asset?

  • Origin: https://github.com/protocolbuffers/protobuf/archive/v33.6.tar.gz
  • SHA512/blob name: 16f8689ec7aba47d29f27c2360c33c78d6e11ae9f29e815f792e6b943713395e680f0ab6d48f9395e8bec1df44c4afdc212ad4e4fc3629b820e7b3ac82e132e9
  • Mirror probe: https://vcpkg.storage.devpackages.microsoft.io/artifacts/16f8689ec7aba47d29f27c2360c33c78d6e11ae9f29e815f792e6b943713395e680f0ab6d48f9395e8bec1df44c4afdc212ad4e4fc3629b820e7b3ac82e132e9 currently returns HTTP 404.

Once the probe returns 200, the failed vcpkg checks can be rerun.

This was referenced Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Update to a version of Protobuf greater than or equal to v33.6

6 participants