ci: merge binary license checks into build workflow by Yicong-Huang · Pull Request #4597 · apache/texera

Yicong-Huang · 2026-05-01T06:16:13Z

What changes were proposed in this PR?

This PR merges the standalone binary dependency license checks into the existing Build workflow.

Instead of running a separate Check binary dependency licenses workflow with four extra jobs, the license checks now run inside the existing build jobs:

npm license validation runs in the Linux frontend job after format:ci and the production build, before frontend unit tests.
JVM license validation runs in the scala job after the required database setup and dist build, before the remaining Scala compile/test steps.
Python license validation runs in the Python 3.12 matrix entry immediately after dependency installation, before the PostgreSQL setup and test steps.
agent-service license validation runs in the Linux agent-service job after a production-only install, before the full development install and the lint/typecheck/test steps.

The dedicated .github/workflows/check-binary-licenses.yml workflow is removed so PR checks are less noisy while keeping the underlying validations.

Any related issues, documentation, discussions?

Closes #4596

How was this PR tested?

CI

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Codex GPT-5

aglinxinyuan · 2026-05-01T06:49:44Z

+      - name: Check bundled npm packages against LICENSE-binary
+        if: matrix.os == 'ubuntu-latest'
+        run: ./bin/licensing/check_binary_deps.py npm frontend/dist/3rdpartylicenses.json
+      - name: Run frontend unit tests


Why do we want to change the order of the frontend unit tests here?

aglinxinyuan · 2026-05-01T06:50:40Z

          if [ -f amber/operator-requirements.txt ]; then pip install -r amber/operator-requirements.txt; fi
+          if [ "${{ matrix.python-version }}" = "3.12" ]; then pip install pip-licenses; fi
+      - name: Generate pip-licenses manifest
+        if: matrix.python-version == '3.12'


we pick one Python to check. could be any one. 3.13 is too slow (CI). so maybe we can keep 3.12?

we could also check all pythons. WDYT?

@bobbai00 can decide.

I am fine with just using 3.12.

bobbai00

LGTM

Backport of #4597 to `release/v1.1.0-incubating`. ### What changes were proposed in this PR? This PR merges the standalone binary dependency license checks into the existing `Build` workflow. Instead of running a separate `Check binary dependency licenses` workflow with four extra jobs, the license checks now run inside the existing build jobs: - npm license validation runs in the Linux `frontend` job after the production build. - JVM license validation runs in the `scala` job after backend tests. - Python license validation runs in the Python 3.12 matrix entry. - agent-service license validation runs in the Linux `agent-service` job. The dedicated `.github/workflows/check-binary-licenses.yml` workflow is removed so PR checks are less noisy while keeping the underlying validations. ### Any related issues, documentation, discussions? Cherry-pick of c4adf7f from #4597. Closes #4596 ### How was this PR tested? CI on the release branch. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Codex GPT-5

apache#4521 had the python dep install in the scala and python matrix jobs on `uv pip install --system` for speed. apache#4597 unintentionally rewrote those lines back to stock pip while inlining the binary license checks; the regression has been carried forward by every subsequent rebase. Restore uv for speed. The python job's 3.12 leg is the only one that drives the binary license check (`pip-licenses` -> `check_binary_deps.py python`). Keep stock pip on that leg so the resolved versions match `amber/LICENSE-binary-python`, which is generated with pip and tracks what the production image installs. uv and pip can resolve unpinned transitives differently; without this carve-out the check would false- positive on resolver drift, and we'd be forced to update LICENSE- binary-python to chase the CI side (production still uses pip). Other python legs (3.10, 3.11, 3.13) use uv. The scala job's binary license check is jar-only, so it uses uv too. Dev deps install runs post-snapshot so it can use uv on all legs. Closes apache#4635

…4636) ## What changes were proposed in this PR? #4521 had the python dep install in the scala and python matrix jobs on `uv pip install --system` for install-speed. #4597 unintentionally rewrote those lines back to stock `pip install` while inlining the binary license checks, and the regression has been carried forward by every subsequent rebase. Restore uv — but with a targeted carve-out for the leg that drives the binary license check. ### Why the carve-out The python job's `3.12` matrix entry is the only leg that runs `pip-licenses` and feeds the result into `bin/licensing/check_binary_deps.py python`. That tool compares the installed Python tree against `amber/LICENSE-binary-python`, which is generated **with pip** and tracks what the production image installs. uv and pip resolvers can land on different versions of unpinned transitives — if the 3.12 leg installs with uv, `check_binary_deps.py` would false-positive on resolver drift, forcing us to chase those drifts in `LICENSE-binary-python` (and diverge from production). So: stock `pip install` on the 3.12 leg only; uv everywhere else. ### Per-step shape - **scala job → Install dependencies**: `uv pip install --system`. Its license check is jar-only, so Python resolver differences don't matter here. - **python job → Install dependencies**: branches on `matrix.python-version`. `3.12` keeps `pip install`; `3.10`, `3.11`, `3.13` use `uv pip install --system`. - **python job → Install dev dependencies**: `uv pip install --system`. Runs post-snapshot, so uv is safe on all legs. No behaviour change for the license check itself. Other legs gain install speed. ## Any related issues, documentation, discussions? Closes #4635. Restores #4521. Regression introduced by #4597. ## How was this PR tested? Will be exercised by this PR's own scala and python matrices. The expected signal: - [x] scala job: install step uses uv, tests still run. - [x] python 3.10 / 3.11 / 3.13 legs: install step uses uv. - [x] python 3.12 leg: install step uses pip; pip-licenses manifest unchanged; `check_binary_deps.py python` passes. ## Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Opus 4.7 (Claude Code)

…4636) ## What changes were proposed in this PR? #4521 had the python dep install in the scala and python matrix jobs on `uv pip install --system` for install-speed. #4597 unintentionally rewrote those lines back to stock `pip install` while inlining the binary license checks, and the regression has been carried forward by every subsequent rebase. Restore uv — but with a targeted carve-out for the leg that drives the binary license check. ### Why the carve-out The python job's `3.12` matrix entry is the only leg that runs `pip-licenses` and feeds the result into `bin/licensing/check_binary_deps.py python`. That tool compares the installed Python tree against `amber/LICENSE-binary-python`, which is generated **with pip** and tracks what the production image installs. uv and pip resolvers can land on different versions of unpinned transitives — if the 3.12 leg installs with uv, `check_binary_deps.py` would false-positive on resolver drift, forcing us to chase those drifts in `LICENSE-binary-python` (and diverge from production). So: stock `pip install` on the 3.12 leg only; uv everywhere else. ### Per-step shape - **scala job → Install dependencies**: `uv pip install --system`. Its license check is jar-only, so Python resolver differences don't matter here. - **python job → Install dependencies**: branches on `matrix.python-version`. `3.12` keeps `pip install`; `3.10`, `3.11`, `3.13` use `uv pip install --system`. - **python job → Install dev dependencies**: `uv pip install --system`. Runs post-snapshot, so uv is safe on all legs. No behaviour change for the license check itself. Other legs gain install speed. ## Any related issues, documentation, discussions? Closes #4635. Restores #4521. Regression introduced by #4597. ## How was this PR tested? Will be exercised by this PR's own scala and python matrices. The expected signal: - [x] scala job: install step uses uv, tests still run. - [x] python 3.10 / 3.11 / 3.13 legs: install step uses uv. - [x] python 3.12 leg: install step uses pip; pip-licenses manifest unchanged; `check_binary_deps.py python` passes. ## Was this PR authored or co-authored using generative AI tooling? (backported from commit a3d43db) Generated-by: Claude Opus 4.7 (Claude Code)

ci: merge binary license checks into build workflow

9a64f17

github-actions Bot assigned Yicong-Huang May 1, 2026

github-actions Bot added feature ci changes related to CI labels May 1, 2026

github-actions Bot added 3 commits April 30, 2026 23:20

ci: run agent-service license check before dev install

e2c6ad4

ci: move license checks earlier in build jobs

9749a1b

ci: restore frontend lint before prod build

7ba8556

Yicong-Huang requested a review from bobbai00 May 1, 2026 06:36

Yicong-Huang enabled auto-merge (squash) May 1, 2026 06:36

Merge branch 'main' into feat/merge-license-checks-into-build

d7a2b38

Yicong-Huang requested review from aglinxinyuan May 1, 2026 06:40

Merge branch 'main' into feat/merge-license-checks-into-build

8341016

aglinxinyuan reviewed May 1, 2026

View reviewed changes

bobbai00 approved these changes May 1, 2026

View reviewed changes

Yicong-Huang merged commit c4adf7f into apache:main May 1, 2026
13 checks passed

Yicong-Huang mentioned this pull request May 1, 2026

ci(1.1): merge binary license checks into build workflow #4601

Merged

This was referenced May 2, 2026

Restore uv pip install in scala and python CI jobs #4635

Closed

fix(ci): use uv for python deps, keep pip on 3.12 license-check leg #4636

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: merge binary license checks into build workflow#4597

ci: merge binary license checks into build workflow#4597
Yicong-Huang merged 6 commits into
apache:mainfrom
Yicong-Huang:feat/merge-license-checks-into-build

Yicong-Huang commented May 1, 2026 •

edited

Loading

Uh oh!

aglinxinyuan May 1, 2026

Uh oh!

aglinxinyuan May 1, 2026

Uh oh!

Yicong-Huang May 1, 2026

Uh oh!

Yicong-Huang May 1, 2026

Uh oh!

aglinxinyuan May 1, 2026

Uh oh!

bobbai00 May 1, 2026

Uh oh!

Uh oh!

bobbai00 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Yicong-Huang commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this PR?

Any related issues, documentation, discussions?

How was this PR tested?

Was this PR authored or co-authored using generative AI tooling?

Uh oh!

aglinxinyuan May 1, 2026

Choose a reason for hiding this comment

Uh oh!

aglinxinyuan May 1, 2026

Choose a reason for hiding this comment

Uh oh!

Yicong-Huang May 1, 2026

Choose a reason for hiding this comment

Uh oh!

Yicong-Huang May 1, 2026

Choose a reason for hiding this comment

Uh oh!

aglinxinyuan May 1, 2026

Choose a reason for hiding this comment

Uh oh!

bobbai00 May 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bobbai00 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Yicong-Huang commented May 1, 2026 •

edited

Loading