Skip to content

Generate the Jetpack App Store screenshots from the app - #26339

Open
jkmassel wants to merge 12 commits into
jkmassel/ui-tests-targets-structurefrom
jkmassel/app-store-screenshots
Open

jkmassel wants to merge 12 commits into
jkmassel/ui-tests-targets-structurefrom
jkmassel/app-store-screenshots

Conversation

@jkmassel

@jkmassel jkmassel commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Generates the Jetpack App Store screenshots from the app: the UI tests from #26119 take them against the fixtures, in every App Store language, on iPhone, iPad and iPhone Duo, and a compositor frames them below a headline fitted to each language.

Stacked on #26119.

Summary

  • bundle exec fastlane jetpack_screenshots builds the app with the fixtures and runs the new AppStoreScreenshots UI test once per device and language, saving the raw screenshots. Devices: iPhone 18 Pro Max (6.9"), iPad Pro 13-inch (M5) and iPhone Duo (outer and inner displays).
  • bundle exec fastlane create_jetpack_promo_screenshots composes them with Scripts/app-store-screenshots/compose.swift: a title screen, then each screenshot in a device frame below its headline. update_jetpack_metadata_on_app_store_connect with_screenshots:true uploads them, as before.
  • Headlines are fitted to each language: as large as they fit in two lines, one size for the whole set, balanced lines, Japanese and Chinese broken only between phrases, and a serif that ships with macOS for each script.
  • bundle exec fastlane trigger_jetpack_screenshots does the same on CI: .buildkite/app-store-screenshots.yml builds once on the Xcode 27.1 image, runs a job per device and language (51, in a matrix step per device, since a matrix makes at most 50) at a low priority, so they only use Mac agents nothing else is waiting for, and composes whatever was taken into downloadable artifacts.
  • On-screen content is translated too, for languages with a fastlane/appstoreres/jetpack_screenshot_content/<locale>.json: the lane writes translated copies of the fixture files and the app loads them over the originals. The fixtures the UI tests assert on don't change.
  • The fixtures can serve images (WireMock's bodyFileName) and take an overlay. The new images are the screens' header images and photos, and generated placeholder avatars.
  • The headlines are new GlotPress strings: a feature name ("Stats.") and the rest ("Your metrics, wherever you are."), so every language gets its bold lead without translators keeping markup.
  • Removed: the old Jetpack screenshot layout (jetpack_screenshots.json), its assets and strings, and Scripts/fix-screenshots.rb. The WordPress app's screenshots are unchanged.

docs/app-store-screenshots.md covers all of it.

What a reviewer will want to know

The iPhone Duo needs Xcode 27.1. An app built with an older SDK runs letterboxed on its outer display, so the lane stops if the selected Xcode, or a build it would reuse, is older. App Store Connect takes both Duo displays in one set (APP_IPHONE_DUO), which deliver uploads from fastlane 2.241.1. Trunk's fastlane is already newer.

The simulator only models the Duo's outer display. For the inner one, the lane moves the app to a 669×951 pt resizable display with xcrun devicectl device appResize, as unfolding does. XCUITest can't tap there, so the test asks the lane for each inner screenshot through a file and waits. On the Duo the test relaunches the app before each screen, since not every screen lays itself out for the outer display again after unfolding.

Taps soon after a launch retry. On CI's simulators, XCUITest sometimes doesn't tap at all just after the app launches. The test taps the tabs, the Stats and Posts quick actions and the Reader site in the middle of their frames, until their screens appear, up to three times.

The test finds its way by identifiers and the fixtures' content, never the app's text, since it runs in every language. That needed one app change: an accessibility identifier on the Reader post's back button.

Nothing runs on PRs. The test is skipped unless the lane gives it somewhere to save screenshots, and the screenshot pipeline only runs when it's started. A full local run (17 languages × 3 devices) takes about three hours; on CI it's 51 jobs that can run side by side.

The translations are temporary. Every App Store language has headlines and on-screen content that Claude translated, not translators, so the screenshots can be generated in all 17 now. They're placeholders until both go through the translation pipeline: the headlines are replaced the first time download_jetpack_promo_strings runs after GlotPress has them, and the on-screen content has no pipeline yet. The headline English is the current App Store copy.

Screenshots

Taken locally with jetpack_screenshots. Each image is one language and device side by side, as create_jetpack_promo_screenshots writes them to fastlane/jetpack_screenshots_overview. The Japanese and German headlines are stand-ins.

English

English, iPhone 6.9 inch: title screen, Stats, Editor, Reader and Notifications

English, iPad 13 inch: title screen, Stats, Editor, Reader and Notifications

English, iPhone Duo inner display

English, iPhone Duo outer display

Japanese and German, with translated content

Japanese, iPhone, with the posts and notifications translated

German, iPhone, with the posts and notifications translated

Japanese, iPad, with translated content

German, iPad, with translated content

Not in this PR

  • Uploading the screenshots to App Store Connect.
  • Putting the headlines and on-screen content through the translation pipeline, to replace the temporary translations.
  • A manifest for choosing screens, appearance and theme without editing code.
  • Fixing the app retrying a failed image request over 1,000 times, which the image fixtures sidestep here.

Test plan

  • swift test -Xswiftc -DUI_TEST_HTTP_FIXTURES --filter HTTPFixturesTests: 58 tests passed, including the new body file and overlay tests.
  • ruby fastlane/lanes/screenshot_content_helper_test.rb: 10 runs, 0 failures. All fastlane/lanes/*_test.rb pass, and RuboCop reports no offenses in fastlane and the Gemfile.
  • SwiftLint passes on the changed Swift files, which are stable under swift format.
  • Locally, on Generate the WordPress App Store screenshots from the app #26355's branch: jetpack_screenshots passed with Xcode 27.1 on the iPhone Duo in Dutch, Russian and English and on iPhone in English, and with Xcode 27.2 on iPhone in Italian and on iPad in English.
  • The temporary translations are valid for all 16 languages: each language's content generates its fixture overlay with every notification's names and titles in place, and every headline fits within two lines (three for the title) on the iPhone, iPad and both iPhone Duo layouts, with no warnings from the compositor.
  • .buildkite/app-store-screenshots.yml passes Buildkite's schema validation, and trigger_jetpack_screenshots is registered.
  • Full Jetpack screenshot runs on CI: #36365 and #36366, all 54 jobs passed in each, none on a retry. They ran on Generate the WordPress App Store screenshots from the app #26355's branch, which adds the fixes the pipeline needed on CI agents, such as passing the app to each job as an argument, retrying the iPhone Duo's inner display, and retrying the tap on the draft.

@jkmassel jkmassel added Testing Unit and UI Tests and Tooling Tooling Build, Release, and Validation Tools labels Nov 5, 2026
@jkmassel jkmassel self-assigned this Nov 5, 2026
@jkmassel jkmassel added this to the 27.5 milestone Nov 5, 2026
@dangermattic

dangermattic commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator
1 Warning
⚠️ This PR is larger than 500 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.

Generated by 🚫 Danger

@wpmobilebot

wpmobilebot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor
App Icon📲 You can test the changes from this Pull Request in WordPress by scanning the QR code below to install the corresponding build.
App NameWordPress
ConfigurationRelease-Alpha
Build Number35790
VersionPR #26339
Bundle IDorg.wordpress.alpha
Commitd93fc1e
Installation URL5ni6r8lmpe690
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot

wpmobilebot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor
App Icon📲 You can test the changes from this Pull Request in Jetpack by scanning the QR code below to install the corresponding build.
App NameJetpack
ConfigurationRelease-Alpha
Build Number35790
VersionPR #26339
Bundle IDcom.jetpack.alpha
Commitd93fc1e
Installation URL023gigrggpffg
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot

wpmobilebot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Build Failure Analysis

This build has failures. Claude has analyzed them - check the build annotations for details.

@wpmobilebot

Copy link
Copy Markdown
Contributor

Version 27.5 has now entered code-freeze, so the milestone of this PR has been updated to 27.6.

@jkmassel
jkmassel force-pushed the jkmassel/ui-tests-targets-structure branch from 8c4fae5 to 99d5b33 Compare November 20, 2026 23:13
@jkmassel
jkmassel force-pushed the jkmassel/app-store-screenshots branch 2 times, most recently from cae9472 to c6e37c4 Compare November 20, 2026 23:16
@jkmassel
jkmassel force-pushed the jkmassel/ui-tests-targets-structure branch from 99d5b33 to 239cada Compare November 20, 2026 23:16
@jkmassel
jkmassel force-pushed the jkmassel/app-store-screenshots branch 4 times, most recently from 5624dfe to 2126741 Compare November 21, 2026 02:17
@jkmassel
jkmassel force-pushed the jkmassel/ui-tests-targets-structure branch from 3bd7d91 to d41ce3d Compare November 21, 2026 02:58
@jkmassel
jkmassel force-pushed the jkmassel/app-store-screenshots branch 2 times, most recently from 3467180 to 954dc97 Compare November 21, 2026 04:17
@jkmassel
jkmassel marked this pull request as ready for review November 21, 2026 04:48
@jkmassel
jkmassel requested a review from a team as a code owner November 21, 2026 04:48
@jkmassel
jkmassel requested a review from crazytonyli November 21, 2026 04:48
A mapping can now answer with a file from the fixtures' `__files` directory, by naming it in `bodyFileName` as WireMock does. That's how images are served.

A second fixtures directory can also be laid over the first: its mapping files replace the ones at the same paths. The app loads one from the `-ui-test-http-fixtures-overlay` launch argument. The App Store screenshots use one to show the fixtures' content translated, without changing what the UI tests assert on.
Every image request used to get a 404, which the app retries over and over: over 1,000 times for one Reader header image in a single run. These are the Reader posts' header images, the editor draft's photo, and avatars for the notifications and the Reader post's author.

The avatars are generated placeholders, not anyone's Gravatar. Any other Gravatar gets a plain silhouette.
`AppStoreScreenshots` opens Stats, the editor, a Reader post and Notifications against the fixtures, and saves a screenshot of each. It runs in any language, so it finds its way by identifiers and the fixtures' content rather than the app's text, and it handles the iPad's sidebar and the iPhone Duo.

The Duo's inner display is shown by the lane on the Mac, which the test asks for through a file. The test is skipped unless the lane gives it somewhere to save the screenshots.

The Reader post's back button gets an accessibility identifier, since its label changes with the language.
`compose.swift` draws each screenshot in a device frame below its headline, plus a title screen, for the iPhone, the iPad and both iPhone Duo displays.

Each language's headlines are fitted to it: set as large as they fit in two lines, all at one size, with balanced lines. Japanese and Chinese only break between phrases. Scripts New York doesn't cover get a serif font that ships with macOS.
The screenshots' content comes from the fixtures, in English. `ScreenshotContentHelper` writes, for one language, a copy of every fixture file with content to translate, translated from `jetpack_screenshot_content/<locale>.json`.

Notifications mark names and titles in their text with UTF-16 ranges, which are moved to match the translation. A translation that leaves one out is an error.

The Japanese and German translations are drafts, to be reviewed.
- `jetpack_screenshots` builds the app with the fixtures, and runs `AppStoreScreenshots` for each device and language. Devices: iPhone 18 Pro Max, iPad Pro 13-inch and iPhone Duo. For the Duo it also shows and screenshots the inner display, which needs Xcode 27.1.
- `create_jetpack_promo_screenshots` composes them, ready for `update_jetpack_metadata_on_app_store_connect`.
- The headlines are new GlotPress strings: a feature name and the rest of the headline, for each screenshot.

The old Jetpack screenshot layout, its assets and strings, and `Scripts/fix-screenshots.rb` are removed. fastlane goes to 2.241.1 or later, which uploads the iPhone Duo's screenshots.
The Reader stream replaces its cards as images arrive. Tapping a card through its element can then find it with an infinite frame, which raises an exception that fails the test, as it did once on the iPad. The test now reads the card's frame once and taps that point on the screen, and tries again if the post didn't open.
Headlines and on-screen content for the 16 App Store languages besides English, so the screenshots can be generated in all of them now. Claude wrote them, not translators, and they're placeholders until the headlines and content go through the translation pipeline. Every content translation keeps the names and titles the notifications mark, and every headline fits on the iPhone, iPad and both iPhone Duo displays.
`.buildkite/app-store-screenshots.yml` builds the app once on the Xcode 27.1 image, then takes each device's screenshots in a matrix of the 17 App Store languages: 51 jobs, since a matrix makes at most 50. A last job composes whatever was taken, and uploads the composed screenshots and each language's overview.

`bundle exec fastlane trigger_jetpack_screenshots` starts it on the current branch.
The pipeline's 50-odd jobs now only take Mac agents nothing else is waiting for, so they never hold up a PR or trunk build. Everything else runs at the default of 0, and the release pipelines at 1.
The site menu is a SwiftUI list since #26026. Its row identifiers are
on buttons inside the cells, or on an iPad, where the menu is a
sidebar, on static texts, so the test matches a row by its identifier
on any element.
Soon after a launch, on CI's simulators, a tap sometimes doesn't
happen: the Reader and Notifications tabs, and the Stats and Posts
quick actions on the iPhone Duo, failed this way. Each is now tapped
in the middle until its screen appears, up to three times.
@jkmassel
jkmassel force-pushed the jkmassel/app-store-screenshots branch from 954dc97 to 0493336 Compare November 21, 2026 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Testing Unit and UI Tests and Tooling Tooling Build, Release, and Validation Tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants