[backport-cloud_asset_inventory-1.1] Sync CI configuration with main branch - #21193
Open
mrodm wants to merge 1 commit into
Open
Conversation
5 tasks
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings affect workflow execution, synchronization, backport correctness, and CI tooling.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Syncs CI configuration and backport tooling from main into backport-cloud_asset_inventory-1.1, including Buildkite/GitHub automation and standalone backport utilities.
Changes:
- Synchronizes CI pipelines, workflows, actions, and scripts.
- Adds backport package, ownership, changelog, and validation tooling.
- Removes legacy test reporting code, fixtures, and obsolete dependencies.
Review findings:
- Critical: Several workflows invoke missing scripts or Mage targets.
- Critical: Changelog synchronization, backport target validation, cherry-pick handling, and release commit selection contain correctness issues.
- Moderate: Branch synchronization, owner enforcement, package detection, changelog collection/commenting, and branch-name validation need fixes.
File summaries
| File | Summary |
|---|---|
magefile.go |
Updates root Mage tooling. |
go.mod |
Removes obsolete dependencies. |
dev/testsreporter/xunit.go |
Removes the legacy XML parser. |
dev/testsreporter/testdata/sql_system_1718901944954979316.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/nats_1718676056733968706.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/fortinet_fortigate_1718675058322143368.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/fortinet_fortigate_1718675052813006253.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/elastic_package_registry_1718676502022628094.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/CODEOWNERS-default-tests |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/cisco_umbrella_1718676502022628093.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/cisco_umbrella_1718675095255991383.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/cisco_umbrella_1718675095080627646.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/testdata/cisco_umbrella_1718675092421310216.xml |
Removes a legacy reporter fixture. |
dev/testsreporter/packageerror.go |
Removes legacy package error handling. |
dev/testsreporter/packageerror_test.go |
Removes legacy reporter tests. |
dev/testsreporter/githubissue.go |
Removes legacy issue integration. |
dev/testsreporter/github.go |
Removes legacy GitHub integration. |
dev/testsreporter/format.go |
Removes legacy formatting logic. |
dev/testsreporter/errorlinks.go |
Removes legacy error-link handling. |
dev/testsreporter/dataerror.go |
Removes legacy data error handling. |
dev/testsreporter/builderror.go |
Removes legacy build error handling. |
dev/testsreporter/builderror_test.go |
Removes legacy reporter tests. |
dev/testsreporter/_static/summary.tmpl |
Removes a legacy reporter template. |
dev/testsreporter/_static/description.tmpl |
Removes a legacy reporter template. |
dev/scripts/README.md |
Documents development scripts. |
dev/scripts/get_release_commit.sh |
Finds package release commits. |
dev/scripts/backport_apply.sh |
Wraps the backport CLI. |
dev/packagenames/testdata/no_duplicates/p2/manifest.yml |
Provides unique-name test data. |
dev/packagenames/testdata/no_duplicates/p1/manifest.yml |
Provides unique-name test data. |
dev/packagenames/testdata/nested/no_duplicates/technology/p2/manifest.yml |
Provides nested unique-name test data. |
dev/packagenames/testdata/nested/no_duplicates/technology/p1/manifest.yml |
Provides nested unique-name test data. |
dev/packagenames/testdata/nested/no_duplicates/p3/manifest.yml |
Provides nested unique-name test data. |
dev/packagenames/testdata/nested/duplicates/technology/p2/manifest.yml |
Provides nested duplicate-name test data. |
dev/packagenames/testdata/nested/duplicates/technology/p1/manifest.yml |
Provides nested duplicate-name test data. |
dev/packagenames/testdata/nested/duplicates/p3/manifest.yml |
Provides nested duplicate-name test data. |
dev/packagenames/testdata/invalid_manifests/p2/manifest.yml |
Provides invalid-manifest test data. |
dev/packagenames/testdata/invalid_manifests/p1/manifest.yml |
Provides invalid-manifest test data. |
dev/packagenames/testdata/duplicates/p2/manifest.yml |
Provides duplicate-name test data. |
dev/packagenames/testdata/duplicates/p1/manifest.yml |
Provides duplicate-name test data. |
dev/packagenames/packagenames.go |
Validates unique package names. |
dev/packagenames/packagenames_test.go |
Tests package-name validation. |
dev/import-beats/README.md |
Updates import documentation. |
dev/gitutil/git.go |
Provides Git helpers. |
dev/codeowners/testdata/test_packages/package_1/manifest.yml |
Provides ownership test data. |
dev/codeowners/testdata/nested_packages/package_top/manifest.yml |
Provides nested ownership test data. |
dev/codeowners/testdata/nested_packages/category/package_nested_2/manifest.yml |
Provides nested ownership test data. |
dev/codeowners/testdata/nested_packages/category/package_nested_1/manifest.yml |
Provides nested ownership test data. |
dev/codeowners/testdata/nested_packages/category/package_nested_1/data_stream/stream_2/.keep |
Preserves a nested test fixture. |
dev/codeowners/testdata/nested_packages/category/package_nested_1/data_stream/stream_1/.keep |
Preserves a nested test fixture. |
dev/codeowners/testdata/devexp/manifest.yml |
Provides ownership test data. |
dev/codeowners/testdata/CODEOWNERS-owners-trailing-slash |
Tests trailing-slash handling. |
dev/codeowners/testdata/CODEOWNERS-nested-valid |
Tests valid nested ownership. |
dev/codeowners/testdata/CODEOWNERS-nested-streams-valid |
Tests valid nested stream ownership. |
dev/codeowners/testdata/CODEOWNERS-nested-streams-missing-owners |
Tests missing stream owners. |
dev/codeowners/testdata/CODEOWNERS-nested-missing-owner |
Tests missing nested owners. |
dev/codeowners/testdata/CODEOWNERS-nested-category-owner |
Tests category ownership. |
dev/citools/subscription.go |
Provides subscription helpers. |
dev/citools/packages.go |
Discovers packages. |
dev/citools/packagemanifest.go |
Parses package manifests. |
dev/citools/logsdb.go |
Provides LogsDB helpers. |
dev/citools/logsdb_test.go |
Tests LogsDB helpers. |
dev/citools/kibana.go |
Provides Kibana helpers. |
dev/citools/kibana_test.go |
Tests Kibana helpers. |
dev/citools/gomod.go |
Provides Go module helpers. |
dev/citools/gomod_test.go |
Tests Go module helpers. |
cmd/backport/tools.go |
Declares CLI development tools. |
cmd/backport/magefile.go |
Defines CLI tasks. |
cmd/backport/go.mod |
Defines the standalone CLI module. |
cmd/backport/gitutil/git.go |
Provides CLI Git helpers. |
cmd/backport/citools/packages.go |
Discovers CLI packages. |
cmd/backport/citools/packagemanifest.go |
Parses CLI package manifests. |
cmd/backport/backports/packages/detect.go |
Detects changed packages. |
cmd/backport/backports/owners/compare_integration_test.go |
Tests ownership comparison integration. |
cmd/backport/backports/owners/check.go |
Checks owner synchronization. |
cmd/backport/backports/owners/check_test.go |
Tests owner checks. |
cmd/backport/backports/changelog/update.go |
Updates changelog links. |
cmd/backport/backports/changelog/update_test.go |
Tests changelog updates. |
cmd/backport/backports/changelog/sync_test.go |
Tests changelog synchronization. |
cmd/backport/backports/changelog/resolve.go |
Resolves package directories. |
cmd/backport/backports/changelog/resolve_test.go |
Tests package resolution. |
cmd/backport/backports/changelog/insert.go |
Inserts changelog entries. |
cmd/backport/backports/changelog/insert_test.go |
Tests changelog insertion. |
cmd/backport/backports/changelog/extract.go |
Extracts changelog entries. |
cmd/backport/backports/changelog/extract_test.go |
Tests changelog extraction. |
cmd/backport/backports/changelog/comment.go |
Builds synchronization comments. |
cmd/backport/.go-version |
Pins the CLI Go version. |
.gitignore |
Updates ignored artifacts. |
.github/workflows/validate-yaml-dashboards.requirements.txt |
Updates dashboard validation requirements. |
.github/workflows/validate-package-docs.yml |
Validates package documentation. |
.github/workflows/vale-report.yml |
Reports Vale results. |
.github/workflows/vale-lint.yml |
Runs Vale linting. |
.github/workflows/updatecli/updatecli.d/sync-packages-to-bug-issue-template.yml |
Synchronizes issue-template package data. |
.github/workflows/trigger-text-auditor.yml |
Triggers text auditing. |
.github/workflows/trigger-stale-issues.yml |
Triggers stale-issue handling. |
.github/workflows/trigger-pr-review.yml |
Triggers PR review automation. |
.github/workflows/trigger-pr-actions-detective.yml |
Triggers PR diagnostics. |
.github/workflows/trigger-package-tests-security-ml.yml |
Triggers package tests. |
.github/workflows/trigger-newbie-contributor-patrol.yml |
Triggers contributor patrol. |
.github/workflows/trigger-mention-in-pr.yml |
Handles PR mentions. |
.github/workflows/trigger-mention-in-issue.yml |
Handles issue mentions. |
.github/workflows/trigger-issue-triage.yml |
Triggers issue triage. |
.github/workflows/trigger-duplicate-issue-detector.yml |
Triggers duplicate detection. |
.github/workflows/trigger-docs-patrol.yml |
Triggers documentation patrol. |
.github/workflows/trigger-bug-hunter.yml |
Triggers bug detection. |
.github/workflows/trigger-breaking-change-detector.yml |
Triggers breaking-change detection. |
.github/workflows/sync-backport-changelog.yml |
Synchronizes backport changelogs. |
.github/workflows/sweep-ingest-pipeline-safety.yml |
Runs ingest pipeline checks. |
.github/workflows/sweep-httpjson-pagination.yml |
Runs pagination checks. |
.github/workflows/sweep-field-mapping-conflicts.yml |
Runs field-mapping checks. |
.github/workflows/sweep-dashboard-data-scope.yml |
Runs dashboard checks. |
.github/workflows/requires-update.yml |
Runs required-package updates. |
.github/workflows/pr-buildkite-detective.yml |
Detects Buildkite failures. |
.github/workflows/post-backport-checklist.yml |
Posts backport checklists. |
.github/workflows/notify-package-docs-failure.yml |
Reports documentation failures. |
.github/workflows/docs-preview-cleanup.yml |
Cleans documentation previews. |
.github/workflows/docs-edit-automation.yml |
Automates documentation edits. |
.github/workflows/docs-deploy.yml |
Deploys documentation. |
.github/workflows/docs-build.yml |
Builds documentation. |
.github/workflows/ci-comment.yml |
Posts CI comments. |
.github/workflows/catalog-info.yml |
Synchronizes catalog validation. |
.github/workflows/bump-elastic-stack-version.yml |
Automates dependency version updates. |
.github/workflows/backport-packages-detect.yml |
Detects packages for backporting. |
.github/actions/sync-backport-changelog/action.yml |
Defines changelog synchronization action. |
.buildkite/scripts/trigger_integrations_in_parallel.sh |
Triggers integration jobs. |
.buildkite/scripts/trigger_backport.sh |
Triggers backport jobs. |
.buildkite/scripts/test_one_package.sh |
Runs package tests. |
.buildkite/scripts/test_integrations_with_serverless.sh |
Runs serverless integration tests. |
.buildkite/scripts/test_helpers.sh |
Provides shell test helpers. |
.buildkite/scripts/test_check_backport_owners.sh |
Tests owner reporting. |
.buildkite/scripts/run_dev_scripts_tests.sh |
Runs development script tests. |
.buildkite/scripts/run_buildkite_scripts_tests.sh |
Runs Buildkite script tests. |
.buildkite/scripts/requirements-ci-python-scripts.txt |
Defines CI Python requirements. |
.buildkite/scripts/process_benchmarks.sh |
Processes benchmarks. |
.buildkite/scripts/packages/crowdstrike.sh |
Runs package-specific checks. |
.buildkite/scripts/notify_backport_pr.sh |
Notifies backport PRs. |
.buildkite/scripts/non_package_patterns.txt |
Defines non-package patterns. |
.buildkite/scripts/check_changelog_versions_in_main.sh |
Checks changelog versions. |
.buildkite/scripts/check_backports_inventory.sh |
Validates backport inventory. |
.buildkite/scripts/check_backport_tool.sh |
Validates the backport CLI. |
.buildkite/scripts/build_packages.sh |
Builds packages. |
.buildkite/scripts/backport_branch_lib.sh |
Provides backport branch helpers. |
.buildkite/pull-requests.json |
Updates CI trigger rules. |
.buildkite/pipeline.serverless.yml |
Synchronizes serverless testing. |
.buildkite/pipeline.schedule-weekly.yml |
Synchronizes weekly scheduling. |
.buildkite/pipeline.schedule-daily.yml |
Synchronizes daily scheduling. |
.buildkite/pipeline.publish.yml |
Updates publication configuration. |
.buildkite/pipeline.backport.yml |
Updates backport pipeline configuration. |
.buildkite/pipeline.backport-dispatch.yml |
Adds backport dispatch configuration. |
Review details
Suppressed comments (4)
.buildkite/scripts/backport_branch.sh:172
git rm -r --cachedremoves these paths only from the index and leaves files that exist only on the backport branch in the working tree. The subsequent checkout does not delete those untracked files, andgit add .buildkitestages them again, so the supposedly verbatim sync retains stale CI files (the same pattern is repeated fordevand.githubbelow). Remove the worktree contents as well, or explicitly clean files absent frommain, before staging.
cmd/backport/backports/apply/apply.go:66- This regex does not mirror the inventory validator: it requires a numeric suffix and disallows hyphens, while inventory explicitly accepts branches such as
backport-aws-2024-hotfix(cmd/backport/backports/inventory_test.go:887).applytherefore rejects valid active branches when a full target is supplied (and cannot construct them from2024-hotfix). Reusebackports.ValidateBranchNameor align this regex with the inventory format.
cmd/backport/backports/changelog/collect.go:54 - When another changelog commit is pushed to the same backport PR while this sync PR is still open,
Collectreturns here before examining the new diff. The subsequent push and eventual merge therefore never sync the new entry; update the existing sync branch/PR or otherwise preserve and process the newly collected entries instead of treating an open PR as a permanent skip.
cmd/backport/main.go:249 detect-packagespasses a PR base SHA and head SHA here, butbefore..aftercompares the two endpoint trees. When the PR branch was created before the base advanced, this includes package changes that landed on main after the merge base, so the checklist can contain unrelated packages. Use a three-dot diff for PR detection (the owner-check caller already passes the merge base).
- Files reviewed: 181/190 changed files
- Comments generated: 10
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| PR_AUTHOR: ${{ steps.resolve.outputs.pr_author }} | ||
| COMMENT_ID: ${{ steps.find-comment.outputs.comment_id }} | ||
| BODY_FILE: ${{ steps.find-comment.outputs.body_file }} | ||
| run: .github/scripts/backport/process-checked-branches.sh |
Comment on lines
+68
to
+69
| mage RequiresUpdate | ||
| -dryRun=${{ env.DRY_RUN }} |
Comment on lines
+104
to
+108
| AFTER=$(jq -r '.mergeCommit.oid' <<< "$PR_DATA") | ||
| # Note: for rebase-merged multi-commit PRs, parents[0] is the | ||
| # second-to-last commit — the diff range may be narrower than the | ||
| # original push event. Squash and merge-commit merges are unaffected. | ||
| BEFORE=$(gh api "repos/$REPOSITORY/commits/$AFTER" --jq '.parents[0].sha') || exit 1 |
Comment on lines
+115
to
+118
| - uses: actions/checkout@v7 | ||
| if: steps.resolve.outputs.pr_number != '' | ||
| with: | ||
| fetch-depth: 0 |
Comment on lines
+365
to
+369
| if cherryErr != nil { | ||
| manifestHadConflict, manifestResolved, err := resolveManifestVersionConflict(manifestPath) | ||
| if err != nil { | ||
| a.abortCherryPick() | ||
| return nil, fmt.Errorf("resolving manifest.yml conflict: %w", err) |
Comment on lines
+941
to
+943
| if branchRE.MatchString(target) { | ||
| return target, nil | ||
| } |
| git log --oneline \ | ||
| -G "^version: ['\"]?${ESCAPED_VERSION}['\"]?[[:space:]]*$" \ | ||
| -- "$MANIFEST" \ | ||
| | awk 'NF {print $1}' | sort -u |
| if err != err { | ||
| return fmt.Errorf("failed to parse DRY_RUN value: %w", err) | ||
| } | ||
| // ListPackages lists all packages found under the packages directory. |
| - label: ":busts_in_silhouette: Check backport package owners" | ||
| key: "check-backport-owners" | ||
| command: ".buildkite/scripts/check_backport_owners.sh" | ||
| soft_fail: true |
| // branchExistsOnRemote returns true if the branch exists on the remote. | ||
| func branchExistsOnRemote(workingBranch, repository string) (bool, error) { | ||
| stdout, _, err := gh.Exec("api", | ||
| fmt.Sprintf("repos/%s/branches/%s", repository, workingBranch), |
Contributor
🚀 Benchmarks reportPackage
|
| Data stream | Previous EPS | New EPS | Diff (%) | Result |
|---|---|---|---|---|
audit_events |
5952.38 | 3759.4 | -2192.98 (-36.84%) | 💔 |
signin_attempts |
13333.33 | 10000 | -3333.33 (-25%) | 💔 |
To see the full report comment with /test benchmark fullreport
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Syncs
.buildkiteand.githubCI configuration frommainto thebackport-cloud_asset_inventory-1.1backport branch.This brings automated backport workflows, new CI scripts, Go tooling, and
GitHub Actions that only work when they are present on the target branch itself.
Proposed commit message
```
Sync CI configuration with main branch for backport-cloud_asset_inventory-1.1.
Keep CI pipelines, GitHub Actions, and tooling consistent between active backport
branches and
main, so that automated backport workflows and tooling improvementsland on all supported branches.
WHY:
GitHub Actions resolves workflow files from the branch where the event occurs,
not from
main. Backport-branch workflows (changelog sync, post-backportchecklist, owner checks, auto-backport) must be present on the backport branch
itself to fire correctly.
```
Author's Checklist
main.Related issues