Skip to content

Commit b4cb401

Browse files
alexs-aeroalexs-aero
authored
feat: [INFRA-563] Add Maven GAV Scan and metadata output (#251)
* feat: [INFRA-563] Add Maven GAV Scan and metadata output * feat: [INFRA-563] Add metadata as properties * fix: [INFRA-563] Suppress test for NuGet if not needed * fix: [INFRA-563] Skip sign verify if no package * fix(tests): [INFRA-563] Ignore manifest and name check calls * feat(gav scan): [INFRA-563] Update readme * fix: [INFRA-563] Align workflows and update docs
1 parent d3cbbd0 commit b4cb401

21 files changed

Lines changed: 661 additions & 184 deletions

‎.github/actions/create-release-bundle/README.md‎

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,14 +9,15 @@ Create a JFrog release bundle from one or more builds. Wraps the `create-release
99

1010
## Inputs
1111

12-
| Input | Required | Default | Description |
13-
| ----------------- | -------- | ------------------------------------------------------- | ------------------------------------ |
14-
| `build-names` | Yes | | Comma-separated `name:version` pairs |
15-
| `bundle-name` | Yes | | Release bundle name |
16-
| `version` | Yes | | Release bundle version |
17-
| `jf-project` | Yes | | JFrog project key |
18-
| `dry-run` | No | `false` | Run without creating the bundle |
19-
| `entrypoint-path` | No | `.github/workflows/create-release-bundle/entrypoint.sh` | Path to the entrypoint script |
12+
| Input | Required | Default | Description |
13+
| ---------------------- | -------- | ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
14+
| `build-names` | Yes | | Comma-separated `name:version` pairs |
15+
| `bundle-name` | Yes | | Release bundle name |
16+
| `version` | Yes | | Release bundle version |
17+
| `jf-project` | Yes | | JFrog project key |
18+
| `dry-run` | No | `false` | Run without creating the bundle |
19+
| `bundle-metadata-path` | No | _(empty)_ | Path to `.maven-bundle-metadata.json` from detect-artifacts; when present, applied to the bundle via `jf release-bundle-annotate` after create |
20+
| `entrypoint-path` | No | `.github/workflows/create-release-bundle/entrypoint.sh` | Path to the entrypoint script |
2021

2122
## Example Usage
2223

@@ -46,3 +47,5 @@ steps:
4647
## Combining All Three Bundle Actions
4748
4849
For the full lifecycle (delete, create, promote) in a single job, see the [example composable matrix workflow](../../workflows/example_composable-matrix.yaml).
50+
51+
To forward Maven detection hints onto the release bundle, pass the path from detect-artifacts (e.g. `${{ steps.detect.outputs.bundle-metadata-path }}`) as `bundle-metadata-path` when calling this action or `reusable_create-release-bundle.yaml`.

‎.github/actions/create-release-bundle/action.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,13 @@ inputs:
2323
description: Path to the create-release-bundle entrypoint script (relative to workspace)
2424
required: false
2525
default: .github/workflows/create-release-bundle/entrypoint.sh
26+
bundle-metadata-path:
27+
description: |
28+
Optional path to `.maven-bundle-metadata.json` (e.g. detect-artifacts output `bundle-metadata-path`).
29+
When set and the file exists, key=value pairs from the JSON are applied to the release bundle version using
30+
`jf release-bundle-annotate` after `jf release-bundle-create`. Relative paths are resolved from the job workspace.
31+
required: false
32+
default: ""
2633
runs:
2734
using: composite
2835
steps:
@@ -40,9 +47,14 @@ runs:
4047
if [ "${{ inputs.dry-run }}" = "true" ]; then
4148
dry_run_arg="--dry-run"
4249
fi
50+
meta_arg=()
51+
if [[ -n "${{ inputs.bundle-metadata-path }}" ]]; then
52+
meta_arg=(--bundle-metadata "${{ inputs.bundle-metadata-path }}")
53+
fi
4354
"$entrypoint" \
4455
--project "${{ inputs.jf-project }}" \
4556
--build-names "${{ inputs.build-names }}" \
4657
--bundle-name "${{ inputs.bundle-name }}" \
4758
--version "${{ inputs.version }}" \
59+
"${meta_arg[@]}" \
4860
$dry_run_arg

‎.github/actions/detect-artifacts/README.md‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Detect artifact types
22

3-
Runs [`detect_types.sh`](../../workflows/deploy-artifacts/detect_types.sh): content-based detection for ambiguous archives (npm tarballs, PyPI sdists, Go module zips, Helm charts) plus structured passes for PyPI wheels, NuGet packages, Maven POMs, and Docker bundle metadata, and writes `structured_build_artifacts/` plus a manifest, matching the deploy entrypoint’s detection phase. JFrog/build metadata is not required because detection only inspects file contents and copies into the structured tree.
3+
Runs [`detect_types.sh`](../../workflows/deploy-artifacts/detect_types.sh): content-based detection for ambiguous archives (npm tarballs, PyPI sdists, Go module zips, Helm charts) plus structured passes for PyPI wheels, NuGet packages, Maven POMs, and Docker bundle metadata, and writes `structured_build_artifacts/` plus a manifest, matching the deploy entrypoint’s detection phase. After scanning, it writes **`.maven-bundle-metadata.json`** (unique Maven GAV count, multi-package flag, aggregator presence, flatten heuristics) under the same structured directory. JFrog/build metadata is not required because detection only inspects file contents and copies into the structured tree.
44

55
## Prerequisites
66

@@ -23,8 +23,18 @@ Runner tools used by detectors: `jq`, `tar`, `unzip`, `xmllint` (for Maven POM d
2323
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
2424
| `structured-artifacts-dir` | Relative path to `structured_build_artifacts/` (per-type subdirs and copied files). If `working-dir` is set, this path is rooted under that directory (e.g. `my-job/structured_build_artifacts`). |
2525
| `manifest-path` | Relative path to the TSV manifest (`.manifest`): one row per primary artifact with columns `path` and `type`. Companion files are gathered on disk but not listed here. |
26+
| `bundle-metadata-path` | Relative path to `.maven-bundle-metadata.json` (see table below). |
2627

27-
In the calling workflow, give the `uses` step an `id` (for example `id: detect-artifacts`), then reference `${{ steps.detect-artifacts.outputs.structured-artifacts-dir }}` and `${{ steps.detect-artifacts.outputs.manifest-path }}`. You can also forward them from `jobs.<job_id>.outputs`.
28+
### Fields in `.maven-bundle-metadata.json`
29+
30+
| Property | Description |
31+
| -------------------------- | ------------------------------------------------------------------------------------------------------------- |
32+
| `is_multi_package` | `true` when the tree has more than one distinct Maven GAV (multi-artifact release). |
33+
| `maven_module_count` | Number of unique Maven `(groupId, artifactId, version)` coordinates found in scanned POMs. |
34+
| `maven_aggregator_present` | `true` if any scanned POM is a reactor root (`packaging` `pom` with `<modules>`). |
35+
| `is_flattened` | `true` if any POM looks flattened (flatten-plugin marker and/or resolved, no-`${}` consumer-style heuristic). |
36+
37+
After detection, `.maven-bundle-metadata.json` can be passed to [create-release-bundle](../create-release-bundle/README.md) as `bundle-metadata-path` so those fields are set on the release bundle version (`jf release-bundle-annotate`).
2838

2939
## Example (after collect, before release bundle)
3040

@@ -47,6 +57,7 @@ steps:
4757
run: |
4858
echo "Structured dir: ${{ steps.detect-artifacts.outputs.structured-artifacts-dir }}"
4959
echo "Manifest: ${{ steps.detect-artifacts.outputs.manifest-path }}"
60+
echo "Bundle metadata: ${{ steps.detect-artifacts.outputs.bundle-metadata-path }}"
5061
```
5162
5263
See [`reusable_artifacts-cicd.yaml`](../../workflows/reusable_artifacts-cicd.yaml) for an integrated job between matrix collect and sign.

‎.github/actions/detect-artifacts/action.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ description: |
33
Run content-based artifact type detection (npm / PyPI sdist / Go module archives) via
44
deploy-artifacts/detect_types.sh. Intended after merged build artifacts are available
55
(e.g. collect-build-artifacts) and before downstream packaging such as create-release-bundle.
6+
Also emits Maven bundle metadata JSON (`.maven-bundle-metadata.json`) when any `*.pom` is scanned.
67
inputs:
78
script-path:
89
description: Path to detect_types.sh (relative to workspace unless absolute)
@@ -25,6 +26,11 @@ outputs:
2526
manifest-path:
2627
description: TSV manifest of structured primary files (path and type columns), under structured-artifacts-dir
2728
value: ${{ steps.detect.outputs.manifest-path }}
29+
bundle-metadata-path:
30+
description: |
31+
JSON file with Maven bundle hints (is_multi_package, maven_module_count, maven_aggregator_present,
32+
is_flattened) under structured-artifacts-dir; absent only if jq is missing (then detection fails).
33+
value: ${{ steps.detect.outputs.bundle-metadata-path }}
2834
runs:
2935
using: composite
3036
steps:
@@ -58,3 +64,4 @@ runs:
5864
fi
5965
echo "structured-artifacts-dir=$struct_dir" >>"$GITHUB_OUTPUT"
6066
echo "manifest-path=$struct_dir/.manifest" >>"$GITHUB_OUTPUT"
67+
echo "bundle-metadata-path=$struct_dir/.maven-bundle-metadata.json" >>"$GITHUB_OUTPUT"

‎.github/workflows/artifacts-cicd/tests/test_artifacts_cicd.bats‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,12 @@ setup() {
118118
if [ "${LOCAL_SIGNING:-}" = "true" ]; then
119119
skip "No .nupkg fixtures in local signing mode"
120120
fi
121-
find "$ARTIFACTS_DIR" -name "*.nupkg" | grep -q .
121+
local nupkg
122+
nupkg=$(find "$ARTIFACTS_DIR" -name "*.nupkg" -print -quit 2>/dev/null || true)
123+
if [[ -z "$nupkg" ]]; then
124+
skip "No .nupkg in ARTIFACTS_DIR (dotnet/NuGet matrix not in this pipeline run)"
125+
fi
126+
[[ -f "$nupkg" ]]
122127
}
123128

124129
@test "Mac .pkg artifact is present" {

‎.github/workflows/create-release-bundle/README.md‎

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -8,19 +8,21 @@ This workflow creates JFrog release bundles by bundling one or more builds into
88

99
## Inputs
1010

11-
| Input | Description | Required | Default |
12-
| -------------------- | ----------------------------------------------------------------------------------------------------------- | -------- | ------------------------------- |
13-
| `jf-project` | JFrog Artifactory project name | Yes | - |
14-
| `jf-build-names` | Comma-separated list of `build-name:version` pairs to include (e.g. `"app-build:1.2.3,client-build:2.1.0"`) | Yes | - |
15-
| `jf-bundle-name` | Name for the release bundle | Yes | - |
16-
| `version` | Version of the release bundle | Yes | - |
17-
| `jf-url` | JFrog Artifactory URL | No | `https://artifact.aerospike.io` |
18-
| `oidc-provider-name` | OIDC provider name for authentication | No | `gh-aerospike` |
19-
| `oidc-audience` | OIDC audience for authentication | No | `aerospike` |
20-
| `runs-on` | The runner to use for the build | No | `ubuntu-22.04` |
21-
| `gh-checkout-path` | Directory to checkout the shared-workflows repository into | No | `shared-workflows` |
22-
| `gh-workflows-ref` | Git ref for shared-workflows (**should match `uses:`**) | Yes | - |
23-
| `dry-run` | Whether to run in dry-run mode | No | `false` |
11+
| Input | Description | Required | Default |
12+
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------------------------------- |
13+
| `jf-project` | JFrog Artifactory project name | Yes | - |
14+
| `jf-build-names` | Comma-separated list of `build-name:version` pairs to include (e.g. `"app-build:1.2.3,client-build:2.1.0"`) | Yes | - |
15+
| `jf-bundle-name` | Name for the release bundle | Yes | - |
16+
| `version` | Version of the release bundle | Yes | - |
17+
| `jf-url` | JFrog Artifactory URL | No | `https://artifact.aerospike.io` |
18+
| `oidc-provider-name` | OIDC provider name for authentication | No | `gh-aerospike` |
19+
| `oidc-audience` | OIDC audience for authentication | No | `aerospike` |
20+
| `runs-on` | The runner to use for the build | No | `ubuntu-22.04` |
21+
| `gh-checkout-path` | Directory to checkout the shared-workflows repository into | No | `shared-workflows` |
22+
| `gh-workflows-ref` | Git ref for shared-workflows (**should match `uses:`**) | Yes | - |
23+
| `dry-run` | Whether to run in dry-run mode | No | `false` |
24+
| `bundle-metadata-path` | Optional path to `.maven-bundle-metadata.json` (e.g. detect-artifacts `bundle-metadata-path`). Applied as bundle properties after create. | No | _(empty)_ |
25+
| `gh-bundle-metadata-artifact-name` | When set, downloads this GitHub artifact and uses the contained `.maven-bundle-metadata.json` (e.g. `reusable_deploy-artifacts` output `bundle-metadata-artifact-name`). Overrides `bundle-metadata-path` when both are set. | No | _(empty)_ |
2426

2527
## Example Usage
2628

0 commit comments

Comments
 (0)