Repository navigation
Media: Preserve EXIF metadata when converting HEIC to JPEG client-side - #82040
Conversation
A canvas re-encode produces a JPEG with no metadata, so uploads of HEIC files on servers without HEIC support lost their EXIF-derived title, caption, alt text and image_meta. Extract the EXIF block from the HEIC container and embed it into the converted JPEG as an APP1 segment, with the Orientation tag reset to 1 because the conversion bakes rotation into the pixels. The server can then read the metadata exactly as it does for a direct JPEG upload, with no REST API changes. Fixes #82033 Claude-Session: https://claude.ai/code/session_013z1ivmV3cKxpRtaJpaEXbu
|
Size Change: +447 B (+0.01%) Total Size: 7.92 MB 📦 View Changed
|
There was a problem hiding this comment.
Pull request overview
Preserves EXIF metadata when HEIC images are converted client-side to JPEG.
Changes:
- Extracts EXIF payloads from ISOBMFF containers.
- Resets orientation and embeds EXIF into JPEG APP1 segments.
- Adds unit coverage and changelog documentation.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
packages/upload-media/src/test/exif-embed.ts |
Tests EXIF extraction and JPEG embedding. |
packages/upload-media/src/test/canvas-utils.ts |
Tests metadata preservation during conversion. |
packages/upload-media/src/heic-parser.ts |
Exposes raw EXIF payload extraction. |
packages/upload-media/src/exif-embed.ts |
Implements orientation reset and APP1 embedding. |
packages/upload-media/src/canvas-utils.ts |
Preserves EXIF across conversion strategies. |
packages/upload-media/CHANGELOG.md |
Documents the fix. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
I'm not sure how to review this PR. Do you have links to the specs you worked from? How do we confirm the correctness of In its current form, I feel like the only way I could review this PR is by throwing it at Claude and hoping for the best, which doesn't feel right to me, and I'm a bit concerned about the maintainability of code here that I don't understand. All that said, happy to leave this for other reviewers if there are folks more knowledgeable in this area! |
|
Does vips have some built-in EXIF read & write capabilities? I know it uses libexif. Alternatively, might be worth considering using an existing EXIF library for this kind of data manipulation. Looks like there are plenty: https://github.com/jeremy-code/libexif-wasm |
Good point, we should be able to use libexif as long as it can process the HEIC. I'll give that a try.
I'm going to try reworking on it to avoid that. |
Replace the hand-written EXIF extraction and APP1 splicing with libvips: the browser still decodes the HEIC to pixels, but vips now encodes them and copies the EXIF block over from the container header, resetting the orientation since the pixels are already upright. libvips reads the block without an HEVC decoder, so no binary parsing is needed on our side. The '[unlimited]' load option is required because phone HEICs exceed libheif's default item-reference limit. Only the header is parsed, so the decode-time guard it disables does not apply. Metadata now carries over whenever client-side media processing is active; the canvas-only fallback keeps producing a JPEG without metadata.
Upload a real HEIC carrying camera tags and an Orientation of 6 through the client-side pipeline, then assert that the server read the camera metadata from the converted JPEG with the orientation reset to 1, that the original keeps its EXIF segment, and that every sub-size is stripped, matching core. Playwright's Chromium has no HEVC decoder, so createImageBitmap() is shimmed for image/heic blobs to decode a same-sized JPEG; the HEIC bytes still reach vips untouched, which is where the EXIF is read from.
Trunk now enforces test environments by filename suffix; the node environment is the default project, so the docblock is redundant.
Chromium types an uploaded file from the platform, so the fixture arrives as image/heic on macOS but image/heif on Linux, where the shim missed it and the real HEIC reached a browser with no HEVC decoder. Match the family instead, build the stand-in bitmap source up front rather than fetching a data URL, and report the types the shim saw when the converted image never appears, since the upload store reports every conversion failure as a decode error.
9e91001 to
f1ededc
Compare
|
Flaky tests detected in f1ededc. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/33225094031 should allow user to add and remove multiple local font files in
|
There was a problem hiding this comment.
Thanks for the updates, this is much more readable to me, and re-using vips for copying exif data sounds like a pragmatic path forward 👍
Just a quick comment on:
In Safari on macOS (or Chrome 107+ on macOS, which uses the VideoDecoder path), upload a HEIC photo that carries EXIF data - any iPhone photo works - into an Image block.
I assume this PR is just scoped to the Chrome path? Giving this a quick test in Chrome and Safari I'm finding the EXIF data copy only seems to apply to the generated image in Chrome and not Safari, I think because the latter doesn't support client-side media processing fully yet due to the SharedArrayBuffer thing?
All up, I think adding in this gap (preserving EXIF metadata) makes sense for the current pipeline. As an aside, just checking: do we have an issue for the long-running problem of needing to convert HEIC images to a canvas (and therefore reducing bit-depth etc) rather than being able to natively support it? I see we have #79559 and the earlier #76513, and I figure we're still blocked on the patent/licensing problem, but wondered if it's worth keeping track in a current issue, mostly as it'd be great to somehow ensure we (one day) support HDR there given that we're trying to do so in other contexts (e.g. #82362)
In any case, apologies if that's quite off topic from the PR at hand! Happy to give this PR a closer look next week as I'm off tomorrow.
Oh, thats true now. Will update.
I don't think we do have an issue for that, I can create one. Its possible we can get better web api support that would enable this with native codecs, but you are right on the licensing part: we can't ship an HEIC decoder, we can only rely on whats available in the platform/browser.
No worries, appreciate the thoughts. I'm out Monday for a US Holiday, but I'll check back on it Tuesday my time. |
…eic-exif # Conflicts: # packages/upload-media/CHANGELOG.md # packages/upload-media/src/store/private-actions.ts # packages/vips/CHANGELOG.md
Prettier and the API docs generator changed their output after the trunk merge, and the changelog validator requires a Gutenberg PR link on Unreleased entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ALjuX9zShuLdXnPYZzTDg
|
I updated the PR description to match the functionality and did a little clean up. |
There was a problem hiding this comment.
Testing nicely for me in Chrome! In the below screenshot the left item is the original test.heic file with all its metadata. The middle item is trunk where the converted JPEG loses its EXIF metadata. The right item is this PR, where the EXIF data is preserved, while the colour profile correctly specifies sRGB instead of Display P3 in the original HEIC file:
All camera info, and latitude / longitude is preserved as I'd expect 👍
Just left a couple of nit-picky comments, but nothing major. The only other thing that came up from running this through a Claude session was to double-check the memory consumption of the additional copying of the pixels for the vips encode. It seems fine to me, but just thought I'd mention it in case this change affects our calculations for the desired amount of memory we need to account for.
LGTM, and thanks again for all the back and forth here, glad that we could get vips to do the job copying the exif! 🎉
The EncodePixelsOptions interface was inserted between the docblock and the interface it describes, leaving one with two docblocks and the other with none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F7FJD5pXy7qjC2UVNewWXp
The memory check lived in an inline wrapper in addItem(), but encodeCanvas() already falls back to the canvas encoder when an encoder declines, so vipsEncodeCanvasAsJpeg() can decline on its own and keep the whole decision in one place. It now uses the interlaced budget: the decoded pixels are handed to vips as a raw buffer that can't be shrunk on load, so the whole image sits in wasm memory, as with a progressive JPEG. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F7FJD5pXy7qjC2UVNewWXp
Addressed those and will get this merged. |
The TypeScript test graph check flags the mocked call's options argument as possibly undefined. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F7FJD5pXy7qjC2UVNewWXp
#82040) Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org> Co-authored-by: andrewserong <andrewserong@git.wordpress.org> Co-authored-by: swissspidy <swissspidy@git.wordpress.org>
|
I just cherry-picked this PR to the wp/7.2 branch to get it included in the next release: dfd6194 |
This updates the pinned commit hash of the Gutenberg repository from `be6a50be6834e922b9db12c75c9a2cd817b2db59` to `d72020a651eda64dcc53b5dbd5b9d5bd511c4cc9` (version 24.3.0-rc.1). A full list of changes included in this commit can be found on GitHub: WordPress/gutenberg@be6a50b...d72020a - BorderControl: Remove double focus outline (WordPress/gutenberg#85173) - Snackbar: Use design system focus ring (WordPress/gutenberg#85168) - SelectControl: Fix default options showing the `Select` placeholder (WordPress/gutenberg#85172) - Actions API: Remove the unsupported `single` action context (WordPress/gutenberg#85186) - Client-side media: Upgrade wasm-vips to 0.0.19 and raise memory budgets accordingly (WordPress/gutenberg#84980) - Styles: Fix inconsistent 'Elements' tools panel heading level (WordPress/gutenberg#85247) - `media-utils`/MediaEdit: use design system focus ring (WordPress/gutenberg#84940) - Icons: Add new form based icons. (WordPress/gutenberg#84074) (WordPress/gutenberg#85360) - Navigation: keep custom CSS off inner list containers (WordPress/gutenberg#84881) (WordPress/gutenberg#85367) - Navigation: keep Global Styles custom CSS off inner list containers (WordPress/gutenberg#85366) - Edit as HTML: Show design system focus ring on the HTML textarea (WordPress/gutenberg#85266) (WordPress/gutenberg#85359) - Icons: Restore the missing speech bubble in postComments (WordPress/gutenberg#85275) - Block Directory: Standardize downloadable block focus indicators (WordPress/gutenberg#85307) - Global Styles: Add icons and tweak styling for element and block buttons (WordPress/gutenberg#85363) (WordPress/gutenberg#85414) - Tests: Stop asserting Core lacks the core fields functions (WordPress/gutenberg#85201) - Meta Boxes: Use the design system focus ring for the panel toggle (WordPress/gutenberg#85355) (WordPress/gutenberg#85410) - DataViews: Use the design system focus ring for list items. (WordPress/gutenberg#85358) (WordPress/gutenberg#85435) - Components: Standardize checkbox and radio focus outlines (WordPress/gutenberg#85379) (WordPress/gutenberg#85436) - UI: Use solid brand highlights for popup items (WordPress/gutenberg#84087) - Media: Preserve EXIF metadata when converting HEIC to JPEG client-side (WordPress/gutenberg#82040) - Icons: Add error, info and postComments to the core-admin collection (WordPress/gutenberg#85452) - UnitControl: Use the design system outline for the unit select focus ring (WordPress/gutenberg#85444) - Page List: Keep pages visible in the Navigation sidebar in contentOnly (WordPress/gutenberg#85357) - Icons: Add new asterisk icon (WordPress/gutenberg#85442) (WordPress/gutenberg#85498) - wp-env: Keep Debian security updates in Bullseye-based images (WordPress/gutenberg#85469) - Editor: Guard entity preload filters and share entity actions/fields logic (WordPress/gutenberg#85390) (WordPress/gutenberg#85499) - UI: Scale Spinner stroke width with its size (WordPress/gutenberg#85378) - [WP 7.2] Fix the block-library TypeScript test project (WordPress/gutenberg#85533) - Editor: Vertically center the Categories loading spinner (WordPress/gutenberg#85563) - DataViews: Render table bulk actions outside the table (WordPress/gutenberg#85250) (WordPress/gutenberg#85571) - Record e2e traces only when a failed test is retried (WordPress/gutenberg#85428) - Writing flow: fix triple click in a list item selecting the next item (WordPress/gutenberg#85569) - Build: Limit concurrent wp-build builds (WordPress/gutenberg#79889) - Icons: Promote Dashicon replacements to `core-admin` and map their Dashicons (WordPress/gutenberg#85541) Props peterwilsoncc, mamaduka. See #66340 git-svn-id: https://develop.svn.wordpress.org/trunk@64796 602fd350-edb4-49c9-b593-d223f7449a82
This updates the pinned commit hash of the Gutenberg repository from `be6a50be6834e922b9db12c75c9a2cd817b2db59` to `d72020a651eda64dcc53b5dbd5b9d5bd511c4cc9` (version 24.3.0-rc.1). A full list of changes included in this commit can be found on GitHub: WordPress/gutenberg@be6a50b...d72020a - BorderControl: Remove double focus outline (WordPress/gutenberg#85173) - Snackbar: Use design system focus ring (WordPress/gutenberg#85168) - SelectControl: Fix default options showing the `Select` placeholder (WordPress/gutenberg#85172) - Actions API: Remove the unsupported `single` action context (WordPress/gutenberg#85186) - Client-side media: Upgrade wasm-vips to 0.0.19 and raise memory budgets accordingly (WordPress/gutenberg#84980) - Styles: Fix inconsistent 'Elements' tools panel heading level (WordPress/gutenberg#85247) - `media-utils`/MediaEdit: use design system focus ring (WordPress/gutenberg#84940) - Icons: Add new form based icons. (WordPress/gutenberg#84074) (WordPress/gutenberg#85360) - Navigation: keep custom CSS off inner list containers (WordPress/gutenberg#84881) (WordPress/gutenberg#85367) - Navigation: keep Global Styles custom CSS off inner list containers (WordPress/gutenberg#85366) - Edit as HTML: Show design system focus ring on the HTML textarea (WordPress/gutenberg#85266) (WordPress/gutenberg#85359) - Icons: Restore the missing speech bubble in postComments (WordPress/gutenberg#85275) - Block Directory: Standardize downloadable block focus indicators (WordPress/gutenberg#85307) - Global Styles: Add icons and tweak styling for element and block buttons (WordPress/gutenberg#85363) (WordPress/gutenberg#85414) - Tests: Stop asserting Core lacks the core fields functions (WordPress/gutenberg#85201) - Meta Boxes: Use the design system focus ring for the panel toggle (WordPress/gutenberg#85355) (WordPress/gutenberg#85410) - DataViews: Use the design system focus ring for list items. (WordPress/gutenberg#85358) (WordPress/gutenberg#85435) - Components: Standardize checkbox and radio focus outlines (WordPress/gutenberg#85379) (WordPress/gutenberg#85436) - UI: Use solid brand highlights for popup items (WordPress/gutenberg#84087) - Media: Preserve EXIF metadata when converting HEIC to JPEG client-side (WordPress/gutenberg#82040) - Icons: Add error, info and postComments to the core-admin collection (WordPress/gutenberg#85452) - UnitControl: Use the design system outline for the unit select focus ring (WordPress/gutenberg#85444) - Page List: Keep pages visible in the Navigation sidebar in contentOnly (WordPress/gutenberg#85357) - Icons: Add new asterisk icon (WordPress/gutenberg#85442) (WordPress/gutenberg#85498) - wp-env: Keep Debian security updates in Bullseye-based images (WordPress/gutenberg#85469) - Editor: Guard entity preload filters and share entity actions/fields logic (WordPress/gutenberg#85390) (WordPress/gutenberg#85499) - UI: Scale Spinner stroke width with its size (WordPress/gutenberg#85378) - [WP 7.2] Fix the block-library TypeScript test project (WordPress/gutenberg#85533) - Editor: Vertically center the Categories loading spinner (WordPress/gutenberg#85563) - DataViews: Render table bulk actions outside the table (WordPress/gutenberg#85250) (WordPress/gutenberg#85571) - Record e2e traces only when a failed test is retried (WordPress/gutenberg#85428) - Writing flow: fix triple click in a list item selecting the next item (WordPress/gutenberg#85569) - Build: Limit concurrent wp-build builds (WordPress/gutenberg#79889) - Icons: Promote Dashicon replacements to `core-admin` and map their Dashicons (WordPress/gutenberg#85541) Props peterwilsoncc, mamaduka. See #66340 Built from https://develop.svn.wordpress.org/trunk@64796 git-svn-id: http://core.svn.wordpress.org/trunk@63946 1a063a9b-81f0-0310-95a4-ce76da25c4cd
#82040) Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org> Co-authored-by: andrewserong <andrewserong@git.wordpress.org> Co-authored-by: swissspidy <swissspidy@git.wordpress.org>
This updates the pinned commit hash of the Gutenberg repository from `be6a50be6834e922b9db12c75c9a2cd817b2db59` to `d72020a651eda64dcc53b5dbd5b9d5bd511c4cc9` (version 24.3.0-rc.1). A full list of changes included in this commit can be found on GitHub: WordPress/gutenberg@be6a50b...d72020a - BorderControl: Remove double focus outline (WordPress/gutenberg#85173) - Snackbar: Use design system focus ring (WordPress/gutenberg#85168) - SelectControl: Fix default options showing the `Select` placeholder (WordPress/gutenberg#85172) - Actions API: Remove the unsupported `single` action context (WordPress/gutenberg#85186) - Client-side media: Upgrade wasm-vips to 0.0.19 and raise memory budgets accordingly (WordPress/gutenberg#84980) - Styles: Fix inconsistent 'Elements' tools panel heading level (WordPress/gutenberg#85247) - `media-utils`/MediaEdit: use design system focus ring (WordPress/gutenberg#84940) - Icons: Add new form based icons. (WordPress/gutenberg#84074) (WordPress/gutenberg#85360) - Navigation: keep custom CSS off inner list containers (WordPress/gutenberg#84881) (WordPress/gutenberg#85367) - Navigation: keep Global Styles custom CSS off inner list containers (WordPress/gutenberg#85366) - Edit as HTML: Show design system focus ring on the HTML textarea (WordPress/gutenberg#85266) (WordPress/gutenberg#85359) - Icons: Restore the missing speech bubble in postComments (WordPress/gutenberg#85275) - Block Directory: Standardize downloadable block focus indicators (WordPress/gutenberg#85307) - Global Styles: Add icons and tweak styling for element and block buttons (WordPress/gutenberg#85363) (WordPress/gutenberg#85414) - Tests: Stop asserting Core lacks the core fields functions (WordPress/gutenberg#85201) - Meta Boxes: Use the design system focus ring for the panel toggle (WordPress/gutenberg#85355) (WordPress/gutenberg#85410) - DataViews: Use the design system focus ring for list items. (WordPress/gutenberg#85358) (WordPress/gutenberg#85435) - Components: Standardize checkbox and radio focus outlines (WordPress/gutenberg#85379) (WordPress/gutenberg#85436) - UI: Use solid brand highlights for popup items (WordPress/gutenberg#84087) - Media: Preserve EXIF metadata when converting HEIC to JPEG client-side (WordPress/gutenberg#82040) - Icons: Add error, info and postComments to the core-admin collection (WordPress/gutenberg#85452) - UnitControl: Use the design system outline for the unit select focus ring (WordPress/gutenberg#85444) - Page List: Keep pages visible in the Navigation sidebar in contentOnly (WordPress/gutenberg#85357) - Icons: Add new asterisk icon (WordPress/gutenberg#85442) (WordPress/gutenberg#85498) - wp-env: Keep Debian security updates in Bullseye-based images (WordPress/gutenberg#85469) - Editor: Guard entity preload filters and share entity actions/fields logic (WordPress/gutenberg#85390) (WordPress/gutenberg#85499) - UI: Scale Spinner stroke width with its size (WordPress/gutenberg#85378) - [WP 7.2] Fix the block-library TypeScript test project (WordPress/gutenberg#85533) - Editor: Vertically center the Categories loading spinner (WordPress/gutenberg#85563) - DataViews: Render table bulk actions outside the table (WordPress/gutenberg#85250) (WordPress/gutenberg#85571) - Record e2e traces only when a failed test is retried (WordPress/gutenberg#85428) - Writing flow: fix triple click in a list item selecting the next item (WordPress/gutenberg#85569) - Build: Limit concurrent wp-build builds (WordPress/gutenberg#79889) - Icons: Promote Dashicon replacements to `core-admin` and map their Dashicons (WordPress/gutenberg#85541) Props peterwilsoncc, mamaduka. See #66340 git-svn-id: https://develop.svn.wordpress.org/trunk@64796 602fd350-edb4-49c9-b593-d223f7449a82
Claude rewrote this description to match the vips-based approach:
AI Use
Code and description both written with 🤖 Claude Code, reworked after review to lean on vips instead of hand-built EXIF segments. I will review and test.
🤖 Generated with Claude Code
https://claude.ai/code/session_014ALjuX9zShuLdXnPYZzTDg