Skip to content

Fix PACKIT-5421: Check if package already built with fixed dependency - #833

Open
majamassarini wants to merge 7 commits into
packit:mainfrom
majamassarini:fix-packit-5421-rebuild-check
Open

majamassarini wants to merge 7 commits into
packit:mainfrom
majamassarini:fix-packit-5421-rebuild-check

Conversation

@majamassarini

@majamassarini majamassarini commented Sep 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes PACKIT-5421 by checking if a package was already built with the fixed dependency before recommending rebuild.

What Changed

Core Feature: Already-Built Detection

  • Added check_package_built_with_fixed_dependency() to verify if the latest build used the fixed dependency
  • Searches Jira for completed builds with "Fixed in Build" set
  • Fetches root.log from Brew to confirm actual dependency version used during build
  • Compares dependency EVR (Epoch-Version-Release) to determine if rebuild is needed

Multi-Architecture Verification

  • Fetches root.log from all architectures (x86_64, aarch64, ppc64le, s390x)
  • Normalizes epoch values (explicit 0: vs omitted epoch are equivalent)
  • Verifies all architectures agree on dependency version
  • Different architectures can have different buildroots during long builds

Active Build Handling

  • Searches both closed and active (not yet closed) builds
  • Trusts closed builds immediately for verification
  • Checks active builds before returning "needs rebuild"
  • Requests clarification when only active builds exist (might be rejected before closure)
  • Preserves active builds when closed build verification fails

Infrastructure Failure Handling

  • Raises TransientInfrastructureError for Jira/Koji temporary unavailability
  • Task is retried automatically instead of requesting user clarification
  • Handles: Jira query failures, Koji getBuild() failures, listRPMs() errors
  • Distinguishes transient outages from permanent data issues

Resolution Types

  • ALREADY_FIXED: Package has the fix, needs errata processing (not rebuild)
  • NOT_AFFECTED: CVE doesn't apply to this package (continues to reproducer)
  • Distinguishes "work already done" from "CVE doesn't apply"

Edge Cases - Clarification Requested For:

  1. built_after_fix_no_rootlog: Package built after fix but root.log unavailable (Brew retention policy)
  2. architecture_dependency_conflict: Different architectures show different dependency versions
  3. partial_architecture_coverage: Dependency missing from some architecture logs
  4. active_builds_not_closed: Found builds with Fixed in Build set but not yet closed

🤖 Generated with Claude Code

@qodo-for-packit

Copy link
Copy Markdown

PR Summary by Qodo

Avoid rebuilds when packages already use fixed dependencies

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Detects packages already built with fixed dependencies before recommending CVE rebuilds.
• Resolves confirmed builds as not affected and recommends adding them to errata.
• Preserves the existing buildroot workflow when evidence is missing or checks fail.
Diagram

graph TD
    A["Rebuild triage"] --> B["Jira build search"] --> C["Brew root.log"] --> D["EVR comparison"] --> E{"Fixed dependency used?"}
    E -->|Yes| F["Not affected"]
    E -->|No or unknown| G["Buildroot check"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Query Brew builds directly
  • ➕ Avoids relying on Jira status and Fixed in Build field consistency.
  • ➕ Could identify the actual latest package build independently of tracker ordering.
  • ➖ Requires reliable tag selection for every RHEL stream.
  • ➖ Does not directly provide the Jira issue needed in the resolution message.
2. Compare build timestamps
  • ➕ Simpler and avoids downloading and parsing build logs.
  • ➕ Requires fewer external calls and less format-specific logic.
  • ➖ Only infers dependency usage and can produce false positives.
  • ➖ Does not prove which dependency version entered the buildroot.

Recommendation: Keep the PR's root.log and EVR-based verification because it checks the dependency actually used rather than inferring it from timing. Jira supplies the associated tracker and package NVR, while conservative fallbacks preserve the existing buildroot workflow when evidence is unavailable; direct Brew discovery may be worth considering later if Jira ordering proves unreliable.

Files changed (3) +392 / -1

Bug fix (2) +195 / -1
triage_agent.pyShort-circuit rebuilds already containing the dependency fix +49/-1

Short-circuit rebuilds already containing the dependency fix

• The rebuild verification step now checks the latest package build before testing buildroot readiness. Builds confirmed to use the fixed dependency are resolved as NOT_AFFECTED with an errata recommendation; failures or inconclusive results continue through the existing workflow.

ymir/agents/triage_agent.py

utils.pyInspect package build logs for fixed dependency usage +146/-0

Inspect package build logs for fixed dependency usage

• Adds a utility that finds the latest completed package build in Jira, downloads its Brew root.log across supported architectures, and extracts the installed dependency NVR. It handles gzip content, compares Koji EVRs, and conservatively returns an inconclusive result when lookup or parsing fails.

ymir/common/utils.py

Tests (1) +197 / -0
test_utils.pyCover fixed-dependency build detection scenarios +197/-0

Cover fixed-dependency build detection scenarios

• Adds asynchronous unit tests for equal or newer dependencies, older dependencies, missing builds, and gzipped root.log content. The tests mock Jira, HTTP, and Koji interactions and verify returned issue and build metadata.

ymir/common/tests/unit/test_utils.py

@qodo-for-packit

qodo-for-packit Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

⚠️ 49 lower-priority findings omitted to fit the comment size limit; re-run the review or view the findings in the Qodo portal.

Grey Divider


Action required

1. Partially checked builds appear fixed 🐞 Bug ≡ Correctness 👈 Shift left
Description
check_package_built_with_fixed_dependency assigns e.partial_logs to installed_pkgs_logs but
does not retain which required architectures had missing logs. If one architecture returns 404 while
the fetched logs contain the fixed dependency, verification can return True and triage resolves
the build as ALREADY_FIXED without checking the missing architecture.
Code

ymir/common/utils.py[R1227-1230]

elif e.partial_logs:
# Case 2: Partial logs retrieved — use them for verification
installed_pkgs_logs = e.partial_logs
logger.warning(f"Using partial installed_pkgs.log for {package_nvr}: {e}")
Relevance

●●● Strong

Explicit PR intent is partial-log correctness; missing architecture coverage can falsely produce
ALREADY_FIXED.

PR-#562

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The fetcher attaches successfully fetched logs when another required architecture is missing. The
changed handler continues with those logs; subsequent missing-architecture tracking examines only
logs it received, so the fixed-dependency path can return True. Triage converts True directly to
ALREADY_FIXED.

ymir/common/utils.py[727-735]
ymir/common/utils.py[1227-1230]
ymir/common/utils.py[1286-1297]
ymir/common/utils.py[1366-1374]
ymir/agents/triage_agent.py[1255-1290]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Partial installed-package logs are processed as though they cover every required architecture. This can certify a build as fixed when an unchecked architecture used an older dependency.

## Fix Focus Areas
- ymir/common/utils.py[1227-1230]
- ymir/common/utils.py[1286-1297]

## Recommended Fix
Carry the missing-architecture information alongside the fetched logs. Analyze available logs, but do not return an `ALREADY_FIXED` verdict unless every required architecture is covered; return `partial_architecture_coverage` when coverage remains incomplete. Add a multi-architecture test with one fixed-dependency log and one missing log.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


2. Installed dependencies can be missed ✓ Resolved
Description
_get_known_package_names returns only names from the fixed build's RPM list, without adding
dep_component when that name is absent. If the selected package's build log contains the
source-named dependency but the fixed build lists only differently named subpackages, the parser
never checks that log entry and triage can continue toward an unnecessary rebuild; the new
missing-name test also fails.
Code

ymir/common/utils.py[793]

+        known_names = [rpm.get("name") for rpm in rpms if rpm.get("name")]
Relevance

●●● Strong

Dependency names absent from RPM metadata are skipped; adding dep_component is a direct correctness
fix matching dependency-check safeguards.

PR-#562
PR-#441

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The implementation builds known_names solely from listRPMs; the parser iterates solely over
those names before reporting a missing dependency. The added test supplies a list without golang
and asserts that the result includes it.

ymir/common/utils.py[786-799]
ymir/common/utils.py[915-929]
ymir/common/utils.py[1310-1332]
ymir/common/tests/unit/test_utils.py[1851-1868]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The RPM name list can omit the source component, causing the build-log parser to miss an installed dependency and failing the new test.
## Fix Focus Areas
- ymir/common/utils.py[786-799]
- ymir/common/tests/unit/test_utils.py[1851-1868]
## Recommended Fix
Ensure the returned names include `dep_component` when absent, while retaining the binary names returned by Koji.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Permanent data mismatches error out triage ✗ Dismissed
Description
check_package_built_with_fixed_dependency turns permanent validation failures into
TransientInfrastructureError, and verify_rebuild_buildroot re-raises it. This covers a
fixed-dependency build whose Koji name does not match the LLM-supplied dependency_component
(_get_known_package_names/_compare_build_timestamps return None), and the final `except
Exception` also wraps every unexpected error, including JQL 400s for an unknown component. Each such
ticket aborts the whole workflow and reruns the full LLM triage until max_retries, then gets
ymir_triage_errored. Before this change it went on to the standard buildroot check, which still
fails gracefully on the same inputs.
Code

ymir/agents/triage_agent.py[R1407-1413]

+                except TransientInfrastructureError:
+                    raise
+                except Exception as e:
+                    logger.warning(
+                        f"Error checking if {package} was built with fixed dependency: {e}. "
+                        "Continuing with standard buildroot check."
+                    )
Relevance

●●● Strong

Recent triage precedents accept preventing retry loops and preserving graceful handling of permanent
workflow errors.

PR-#540
PR-#843

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
_get_known_package_names returns None when fixed_build.name != dep_component (logged as possible
data corruption), and the caller turns that None into TransientInfrastructureError (utils.py
1157-1162). _compare_build_timestamps returns None on name mismatch, which also becomes transient
(1150-1154). The outer except Exception wraps any other error as transient (1275-1278). The triage
agent re-raises it (1407-1408). The worker's generic handler then calls retry(), which re-queues
the task and reruns run_workflow (the full LLM analysis) until max_retries, then labels the
ticket TRIAGE_ERRORED (triage_agent.py 2027-2035, 1876-1941). For comparison, the existing
check_build_in_buildroot failure on the same inputs just continues (1422-1424).

ymir/common/utils.py[744-756]
ymir/common/utils.py[1156-1162]
ymir/common/utils.py[1272-1278]
ymir/agents/triage_agent.py[1876-1941]
ymir/agents/triage_agent.py[2027-2035]
ymir/agents/triage_agent.py[1415-1424]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Permanent data or validation problems (name mismatch, unknown Jira component, unexpected exceptions) are raised as `TransientInfrastructureError`. That aborts triage, reruns the LLM analysis until retries run out, and finally marks the ticket errored, where before the code fell back to the standard buildroot check.

## Fix Focus Areas
- ymir/common/utils.py[1156-1162]
- ymir/common/utils.py[1150-1154]
- ymir/common/utils.py[1272-1278]
- ymir/agents/triage_agent.py[1407-1413]

## Recommended Fix
Make helpers like `_get_known_package_names` and `_compare_build_timestamps` tell outages (exceptions, no response) apart from validation failures (name mismatch, missing build). Return a distinct reason for validation failures, for example `None, ..., "dependency_validation_failed"`, and handle it in `verify_rebuild_buildroot` with a clarification or the standard buildroot path. Remove the blanket `except Exception -> TransientInfrastructureError` wrapper so that unexpected errors reach the triage agent's `except Exception` fallback.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View high (13)
4. Koji exceptions can trigger a rebuild ✓ Resolved
Description
The outer exception handler in check_package_built_with_fixed_dependency turns unexpected
exceptions into (False, None, None, None). A Koji lookup that raises during architecture or
dependency metadata retrieval therefore becomes a negative build check, and the triage workflow
continues toward its standard rebuild path instead of retrying.
Code

ymir/common/utils.py[R930-935]

except Exception as e:
logger.exception(
f"Error checking if package was built with fixed dependency: {e}. "
"Falling back to standard rebuild check."
)
return False, None, None, None
Relevance

●●● Strong

Matches the PR’s stated retry intent; recent precedent accepts preventing infrastructure failures
from being swallowed.

PR-#843
PR-#675

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The new function makes _get_koji_build calls without local exception conversion and catches their
exceptions in a final handler that returns False; the caller only re-raises
TransientInfrastructureError.

ymir/common/utils.py[1045-1048]
ymir/common/utils.py[1191-1196]
ymir/common/utils.py[1258-1266]
ymir/agents/triage_agent.py[1309-1315]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Unexpected lookup exceptions are converted into a definitive negative build check.
## Fix Focus Areas
- ymir/common/utils.py[1045-1048]
- ymir/common/utils.py[1191-1196]
- ymir/common/utils.py[1258-1266]
## Recommended Fix
Classify Koji and Jira lookup exceptions as `TransientInfrastructureError` at their call sites and avoid returning `False` for an indeterminate infrastructure failure.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Confirmed fixes fail output validation ✓ Resolved
Description
The new OutputSchema constructor supplies AlreadyFixedData, but TriageOutputSchema.data does
not accept that type. Once the missing definitions are added, a confirmed fixed build still raises a
validation error before it reaches the Jira comment.
Code

ymir/agents/triage_agent.py[R1174-1176]

state.triage_result = OutputSchema(
resolution=Resolution.ALREADY_FIXED,
data=AlreadyFixedData(
Relevance

●●● Strong

New model types must be included in Pydantic unions; accepted precedents address missing validation
safeguards.

PR-#726

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The constructor passes the new data type, while the schema's enumerated data union omits it.

ymir/agents/triage_agent.py[1174-1192]
ymir/common/models.py[568-585]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new already-fixed branch constructs data that the triage output schema rejects.
## Fix Focus Areas
- ymir/agents/triage_agent.py[1174-1176]
- ymir/common/models.py[568-585]
## Recommended Fix
Add `AlreadyFixedData` to the `TriageOutputSchema.data` union and test validation of an already-fixed result.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. A failed lookup can hide newest build ✓ Resolved
Description
_select_highest_evr_build records failed Koji metadata lookups but, whenever at least one
candidate resolves, sorts and returns the successful subset as though it were complete. If the
unavailable candidate is the newest completed build, downstream root-log dependency verification
uses an older build as latest and can recommend duplicate rebuild work or send the package to manual
review based on stale evidence.
Code

ymir/common/utils.py[R593-596]

+        if koji_failures:
+            failed_nvrs = [nvr for nvr, _ in koji_failures]
+            raise TransientInfrastructureError(
+                f"Koji lookup failed for {len(koji_failures)} candidate(s): {failed_nvrs}"
Relevance

●●● Strong

Partial metadata results can select stale builds; recent reliability reviews accept safeguards
against incomplete concurrent results.

PR-#675

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Candidate metadata lookups run concurrently, with exceptions accumulated separately while successful
results are retained; the failure branch raises only when no valid candidates remain. The successful
subset is then sorted, and the caller uses its first entry for dependency verification without
retaining the failed candidate, so a lookup failure can remove the actual highest-EVR build from
consideration.

ymir/common/utils.py[568-604]
ymir/common/utils.py[1005-1030]
ymir/common/utils.py[1005-1035]
ymir/common/tests/unit/test_utils.py[2298-2352]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Candidate selection cannot safely identify the highest build when one or more Koji metadata lookups fail. The current implementation treats any successful subset as complete, allowing an older candidate to hide an unavailable newest completed build and control downstream verification.

## Fix Focus Areas
- ymir/common/utils.py[568-604]
- ymir/common/tests/unit/test_utils.py[2298-2352]

## Recommended Fix
After gathering candidate build metadata, raise `TransientInfrastructureError` whenever any candidate lookup raises, rather than only when every lookup fails, so the helper retries instead of selecting from an incomplete subset. Keep invalid or package-mismatched metadata distinct from infrastructure exceptions, and add a test where the newest candidate raises while an older candidate resolves, asserting that no successful candidate is selected.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


7. Active work can still be duplicated ✗ Dismissed
Description
check_package_built_with_fixed_dependency raises for an active-build query failure only when
closed_candidates is empty, treating the same failed query as usable when any closed candidate
exists. If that closed build predates the dependency fix, the unknown active-build set is ignored
and processing reaches the standard rebuild path, which can duplicate a rebuild already underway.
Code

ymir/common/utils.py[R986-988]

+                raise TransientInfrastructureError(
+                    f"Jira query failed for {package} in {fix_version} - "
+                    f"no closed builds found and active builds query failed"
Relevance

●●● Strong

Ignoring failed active-build queries conflicts with the PR’s duplicate-rebuild prevention intent and
recent infrastructure-failure handling precedent.

PR-#806

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The helper records a failed active query as None but continues whenever a closed candidate exists.
Both negative dependency paths test only the truthiness of active_candidates, so None is treated
like a successful empty result and the caller proceeds into ordinary rebuild handling.

ymir/common/utils.py[980-988]
ymir/common/utils.py[1111-1121]
ymir/common/utils.py[1230-1238]
ymir/agents/triage_agent.py[1275-1289]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
An active-build query failure is tolerated whenever a closed candidate exists, even though an older closed build cannot establish that no newer build is currently active. This can let the standard rebuild workflow duplicate work already underway.

## Fix Focus Areas
- ymir/common/utils.py[980-988]
- ymir/common/utils.py[1111-1121]
- ymir/common/utils.py[1230-1238]

## Recommended Fix
Allow closed-build verification to return success when it proves the package is already fixed, but raise `TransientInfrastructureError` before every negative result when `active_candidates` is `None`. Add coverage for an active-query failure combined with a closed build that uses an older dependency.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


8. Brew outages can trigger rebuilds ✓ Resolved
Description
_fetch_root_log collapses every non-200 response into a missing log, while
check_package_built_with_fixed_dependency suppresses the resulting transient error whenever the
build is older than 90 days. For an older package, an authorization failure, throttling response, or
server outage can therefore enter timestamp fallback and classify the package as needing rebuild
when it predates the dependency fix.
Code

ymir/common/utils.py[R1084-1087]

+                if completion_ts:
+                    cutoff_ts = time.time() - (90 * 24 * 60 * 60)
+                    if completion_ts > cutoff_ts:
+                        raise
Relevance

●●● Strong

Conflating outages with missing logs risks incorrect rebuilds; recent reviews favor surfacing
external-service failures explicitly.

PR-#675
PR-#806

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The fetch helper returns no content for every HEAD status other than 200 and for unsuccessful GET
requests, after which absence from all architectures becomes one generic transient exception. The
caller uses only build age to suppress that exception, and the timestamp fallback returns False
for builds completed before the dependency fix.

ymir/common/utils.py[639-654]
ymir/common/utils.py[683-696]
ymir/common/utils.py[1074-1121]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The retention fallback cannot distinguish a genuinely expired root log from Brew authentication, throttling, or server failures. Suppressing all such errors for older builds can turn an infrastructure outage into a rebuild decision.

## Fix Focus Areas
- ymir/common/utils.py[639-654]
- ymir/common/utils.py[683-696]
- ymir/common/utils.py[1074-1093]

## Recommended Fix
Classify expected absence responses such as 404 or 410 separately from transient HTTP failures. Permit timestamp fallback only for confirmed log absence on an old build, while propagating authentication, throttling, 5xx, timeout, and transport failures as `TransientInfrastructureError`; add tests covering both status classes.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


9. Closed-build outages bypass retries ✓ Resolved
Description
_find_completed_builds_jira leaves the closed-build Jira call unguarded and normalizes malformed
responses to an empty list, while check_package_built_with_fixed_dependency rethrows only
TransientInfrastructureError and converts other exceptions to False. When the closed-build query
times out or returns invalid data before completed candidates are retrieved, execution reaches the
normal buildroot and rebuild-consolidation path rather than the queue retry path, without checking
completed work.
Code

ymir/common/utils.py[R1228-1230]

+    except TransientInfrastructureError:
+        # Re-raise infrastructure errors for task retry
+        raise
Relevance

●●● Strong

Closed Jira failures bypass retry and become false negatives, directly violating the PR’s
infrastructure-failure intent.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The closed-build query is awaited without local exception conversion, and invalid responses are
normalized to an empty list, while only active-query failures are converted to a sentinel. The outer
handler rethrows only TransientInfrastructureError; ordinary Jira exceptions reach the generic
handler, which logs them and returns the safe-negative False result, causing the caller to
continue with its standard buildroot check.

ymir/common/utils.py[441-449]
ymir/common/utils.py[975-977]
ymir/common/utils.py[1228-1236]
ymir/agents/triage_agent.py[1267-1276]
ymir/common/utils.py[441-447]
ymir/common/utils.py[459-473]
ymir/agents/triage_agent.py[1275-1291]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Closed-build Jira query failures and malformed responses are not converted to `TransientInfrastructureError`, so the broad fallback in `check_package_built_with_fixed_dependency()` can treat an unavailable search as evidence that no completed fixed build exists and continue into rebuild processing.

## Fix Focus Areas
- ymir/common/utils.py[441-449]
- ymir/common/utils.py[958-977]
- ymir/common/utils.py[1228-1236]
- ymir/common/tests/unit/test_utils.py[2440-2499]

## Recommended Fix
Validate the closed-build response independently and wrap failures from the closed-build `search_jira_issues` invocation, as well as invalid response types, in `TransientInfrastructureError`; alternatively, have the outer handler translate that specific lookup failure before returning `False`. Preserve the existing behavior of continuing with verified closed candidates when only the active-build query fails, and add tests for a raised closed query and a malformed closed response that assert both trigger retry behavior rather than returning `False`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


10. Koji outages erase completed builds ✓ Resolved
Description
_select_highest_evr_build gathers Koji lookup exceptions as values, skips them, and returns the
same None used when Jira candidates are genuinely invalid, so none of those failures become the
newly retried exception. If all lookups fail while the active Jira query succeeds with no results,
check_package_built_with_fixed_dependency returns False and standard buildroot or rebuild
processing proceeds despite known completed-build tickets.
Code

ymir/common/utils.py[R1004-1006]

+                raise TransientInfrastructureError(
+                    f"Jira query failed for {package} in {fix_version} - "
+                    f"closed builds invalid and active builds query failed"
Relevance

●●● Strong

Koji failures are swallowed as no candidates, bypassing the retry behavior explicitly required by
this PR.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Candidate lookups use return_exceptions=True, skip exceptions, and return None when no
successful metadata remains. The caller only raises for a failed active Jira query; with an empty
successful active result, it returns False, after which triage performs the ordinary buildroot
check.

ymir/common/utils.py[559-582]
ymir/common/utils.py[989-1008]
ymir/agents/triage_agent.py[1275-1285]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Koji exceptions encountered while selecting the highest completed build are discarded, making an infrastructure outage indistinguishable from Jira candidates that genuinely reference no valid builds.

## Fix Focus Areas
- ymir/common/utils.py[559-582]
- ymir/common/utils.py[989-1008]
- ymir/common/tests/unit/test_utils.py[2165-2252]

## Recommended Fix
Make candidate selection preserve whether any Koji lookup failed. If no valid candidate can be selected and at least one lookup failed, raise `TransientInfrastructureError`; return `None` only when all completed lookups conclusively show invalid or mismatched builds, and add a test where every candidate lookup raises.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


11. Missing logs can hide unfixed builds ✓ Resolved
Description
_fetch_root_log discards every architecture response without content and returns only successfully
decoded logs, while check_package_built_with_fixed_dependency records missing dependencies only
among that reduced set. If a built architecture's log is unavailable but the remaining logs contain
the fixed dependency, the package can be resolved as already fixed without validating the
unavailable architecture.
Code

ymir/common/utils.py[R642-645]

+                    if isinstance(result, Exception):
+                        continue
+                    url, content = result
+                    if content:
Relevance

●●● Strong

Explicit all-architecture intent makes discarded responses a correctness gap; partial coverage must
trigger manual verification.

PR-#675

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The fetcher retains only successful content, and the later missing_archs calculation iterates
exclusively over the retained logs. Consequently, the consistency and fixed-result paths have no
representation of an architecture whose log request failed or returned a non-200 response.

ymir/common/utils.py[604-665]
ymir/common/utils.py[1033-1111]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The architecture verification path silently omits unavailable root logs, so agreement among only the fetched logs is treated as complete architecture coverage. A missing log for an architecture that was actually built can therefore conceal an older dependency version.

## Fix Focus Areas
- ymir/common/utils.py[635-665]
- ymir/common/utils.py[1033-1111]

## Recommended Fix
Preserve per-architecture fetch outcomes and determine the architectures produced by the selected build from available build metadata. Return an unknown/manual-verification result when a root log for any produced architecture is unavailable or undecodable; only report the package already fixed after every produced architecture has a parsed dependency version that passes the EVR comparison.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


12. Jira outages can start duplicate builds ✓ Resolved
Description
check_package_built_with_fixed_dependency returns None with the manual-action reason
jira_query_failed, but verify_rebuild_buildroot has no branch for that value and falls through
to the standard buildroot check. When the active-build search fails and no closed candidate is
found, existing work remains unknown while processing can reach rebuild consolidation and recommend
duplicate work if the dependency is available.
Code

ymir/common/utils.py[R939-944]

+            if not closed_candidates:
+                logger.warning(
+                    f"No closed builds found and active builds query failed for {package}. "
+                    f"Cannot determine build status."
+                )
+                return None, None, None, "jira_query_failed"
Relevance

●●● Strong

Explicit Jira failure reason is unhandled, allowing unsafe rebuild fallback during outages; similar
triage routing fixes were accepted.

PR-#555
PR-#540

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The helper explicitly classifies an active-query failure with no closed candidates as an
indeterminate manual-action result, and its unit test expects jira_query_failed. The caller
handles several other manual-action reasons but not this one, so execution reaches the documented
fallback to check_build_in_buildroot, which can return the rebuild-consolidation route when the
dependency is available.

ymir/common/utils.py[929-944]
ymir/agents/triage_agent.py[1310-1331]
ymir/common/utils.py[934-942]
ymir/agents/triage_agent.py[1209-1321]
ymir/common/tests/unit/test_utils.py[2383-2418]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A failed active-build Jira query is returned as an unknown verification result, but the caller does not turn `jira_query_failed` into a clarification. This permits normal rebuild processing despite being unable to determine whether a build is already in progress.

## Fix Focus Areas
- ymir/common/utils.py[934-944]
- ymir/agents/triage_agent.py[1175-1285]

## Recommended Fix
Add an explicit `reason == "jira_query_failed"` branch in `verify_rebuild_buildroot` that produces a `CLARIFICATION_NEEDED` result explaining that active-build discovery could not be completed and requesting a retry or manual Jira verification. Return `comment_in_jira` from that branch rather than falling through to the standard buildroot check, and add an integration-level unit test proving that this helper result does not reach that fallback.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


13. Active-search outages discard fixed builds ✓ Resolved
Description
_find_completed_builds_jira awaits the secondary active-build search without isolating its failure
from the closed results already retrieved. If that query raises after the closed query succeeds, the
outer checker converts the entire operation to false and the caller continues with standard rebuild
processing without inspecting the completed build.
Code

ymir/common/utils.py[R450-453]

+    active_results = await run_tool(
+        "search_jira_issues",
+        available_tools=available_tools,
+        jql=active_jql,
Relevance

●●● Strong

Recent precedents accept isolating asynchronous failures to preserve successful work and prevent
workflow-wide loss.

PR-#675
PR-#726

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
run_tool propagates unavailable-tool and execution failures, while the active query has no local
exception handling despite running after the closed results are fetched. The encompassing checker
catches that exception and returns false with no issue or build information, and the triage caller
falls through on false to its standard buildroot and rebuild flow.

ymir/common/utils.py[111-128]
ymir/common/utils.py[432-458]
ymir/common/utils.py[1079-1084]
ymir/agents/triage_agent.py[1314-1327]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Failure of the supplemental active-build Jira query discards successfully retrieved closed builds and can send an already-fixed package into rebuild processing.

## Fix Focus Areas
- ymir/common/utils.py[432-458]
- ymir/common/utils.py[1079-1084]

## Recommended Fix
Catch failures from the active-build query independently of the closed-build query. Preserve and inspect valid closed candidates, while retaining an unknown-active-state signal so a closed build that is not confirmed fixed results in clarification rather than an automatic rebuild path.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


14. Active rebuild work can be duplicated ✓ Resolved
Description
check_package_built_with_fixed_dependency() enters its active-build clarification branch only when
closed_candidates is empty and otherwise evaluates only the selected highest closed build. When
that closed build still uses the old dependency while a newer build is active, the helper returns
False and the caller proceeds through check_build_in_buildroot() toward the ordinary rebuild
workflow instead of requesting clarification about work already underway.
Code

ymir/common/utils.py[920]

+        if not closed_candidates and active_candidates:
Relevance

●●● Strong

Finding exposes a concrete duplicate-work path contradicting the PR’s stated active-build
clarification intent.

PR-#610
PR-#785

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The helper collects active candidates, but its active-build guard is restricted to the case where
there are no closed candidates; when a valid closed candidate exists, it selects from the closed
candidates and can return False if that older build used the vulnerable dependency. The caller
does not handle that result specially and continues into check_build_in_buildroot(), while the
later active-build fallback is not reached in this path.

ymir/common/utils.py[918-940]
ymir/common/utils.py[1057-1071]
ymir/agents/triage_agent.py[1320-1326]
ymir/common/utils.py[488-514]
ymir/common/utils.py[918-929]
ymir/common/utils.py[1060-1071]
ymir/agents/triage_agent.py[1314-1327]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`check_package_built_with_fixed_dependency()` ignores relevant active builds whenever it finds a valid closed candidate. An older closed build can fail dependency verification even though a newer rebuild is already in progress, causing triage to proceed toward duplicate rebuild work rather than request clarification.

## Fix Focus Areas
- ymir/common/utils.py[918-939]
- ymir/common/utils.py[948-1071]

## Recommended Fix
Evaluate the selected closed build normally and preserve an immediate already-fixed result when it independently proves that no rebuild is needed. If the closed build is not confirmed fixed and `active_candidates` is non-empty, compare the active candidates with the selected closed build; when an active build is newer, or recency cannot be established safely, return the existing `active_builds_not_closed:` manual-clarification result instead of returning `False` and continuing into the rebuild workflow.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


15. Five timestamp fallback tests fail ✓ Resolved
Description
_compare_build_timestamps now returns None for unavailable or invalid metadata, while five unit
tests still assert that these cases return False. Missing either build, missing completion data,
or either package-name mismatch therefore fails the suite before the caller can exercise the new
unknown-result path.
Code

ymir/common/utils.py[R827-829]

+    if not package_build or not fixed_build:
+        logger.warning("Could not fetch build metadata for timestamp comparison")
+        return None
Relevance

●●● Strong

Tests assert the obsolete False contract while implementation intentionally returns None for unknown
metadata states.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
All metadata-unavailable and validation-failure branches now return None, and the caller
explicitly converts that value into timestamp_comparison_failed; the existing tests for those five
branches still require False.

ymir/common/utils.py[827-852]
ymir/common/utils.py[959-968]
ymir/common/tests/unit/test_utils.py[1843-1937]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The timestamp helper now distinguishes unavailable or invalid metadata from evidence that a package predates the fix, but five tests retain the old `False` expectations.

## Fix Focus Areas
- ymir/common/tests/unit/test_utils.py[1843-1937]
- ymir/common/utils.py[827-852]

## Recommended Fix
Update the five affected test names, docstrings, and assertions to expect `None`. Preserve `False` only for valid metadata proving that the package build completed before the dependency build.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


16. Three build-log tests fail ✓ Resolved
Description
_fetch_root_log now returns a list of URL/content tuples on success and an empty list when no logs
exist, while its existing unit tests still treat the result as a string or None. The success and
gzip cases fail their string-membership assertions, and the no-log case fails its is None
assertion, breaking the unit suite.
Code

ymir/common/utils.py[631]

+                            decoded_logs.append((url, content.decode("utf-8", errors="replace")))
Relevance

●●● Strong

Tests directly contradict the helper’s new list return contract, causing clear unit failures
requiring test updates.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The helper appends (url, decoded_content) tuples and returns the resulting list, while the tests
perform direct string membership against that list; its unavailable path returns [], while the
corresponding test requires None.

ymir/common/utils.py[619-644]
ymir/common/tests/unit/test_utils.py[1465-1531]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The updated `_fetch_root_log` contract returns architecture-tagged log tuples and uses an empty list for unavailable logs, but three tests still assert the previous string-or-None contract.

## Fix Focus Areas
- ymir/common/tests/unit/test_utils.py[1465-1531]
- ymir/common/utils.py[619-644]

## Recommended Fix
Update the success and gzip tests to inspect the returned tuple contents, and change the unavailable-log test to expect an empty list. Keep the invalid-NVR test expecting `None` only if that distinction is intentional.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 8 rules
Review mode: Auto: ⚖️ Balanced: Substantial dependency-resolution logic changes create real behavioral risk.

Grey Divider

Tip of the day
💡 Did you know, you can show, collapse, or hide each part of a finding: code, evidence, and all

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 477a694 to 6656a92 Compare September 17, 2026 13:47
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 6656a92

@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 6656a92 to 0e07871 Compare September 18, 2026 07:05
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 0e07871

@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 0e07871 to 1750c9c Compare September 18, 2026 07:23
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

Comment thread ymir/common/tests/unit/test_utils.py Outdated
@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 1750c9c

@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 1750c9c to 41d6358 Compare September 18, 2026 07:36
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 41d6358

@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 41d6358 to 0c2c323 Compare September 18, 2026 07:48
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 0c2c323

@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 0c2c323 to b111e6c Compare September 18, 2026 08:10
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/tests/unit/test_utils.py Outdated
@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit b111e6c

@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch 2 times, most recently from 01988a5 to 0d52213 Compare September 18, 2026 08:23
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 35c6a51 to 2e45fdd Compare September 29, 2026 13:54
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 2e45fdd

@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch 2 times, most recently from 142bee6 to 9495861 Compare October 6, 2026 12:00
Comment thread docs/rebuild-decision-logic.md
Comment thread docs/rebuild-decision-logic.md Outdated
Comment thread docs/rebuild-decision-logic.md Outdated
Comment thread docs/rebuild-decision-logic.md Outdated
Comment thread ymir/common/utils.py Outdated
@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 9495861 to 3c27d9f Compare October 9, 2026 08:24
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

Comment thread ymir/common/utils.py
Comment thread ymir/agents/triage_agent.py
Comment thread ymir/common/utils.py Outdated
@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 3c27d9f

Comment thread ymir/common/utils.py Outdated
@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 3c27d9f to 2f9fe55 Compare October 9, 2026 10:05
Ymir was incorrectly recommending rebuild when a package was already
built with the fixed dependency version. Now checks if the package's
current build already used the fix before recommending rebuild.

Implementation uses two verification methods:

1. root.log inspection (primary): Downloads build logs from Brew to see
   which exact dependency NVR was installed. Most reliable since it shows
   what actually happened during the build. Handles epochs, subpackages,
   and concurrent architecture probes.

2. Timestamp comparison (fallback): When root.log unavailable (logs expire
   after some time), compare package completion time vs dependency fix time.
   Use timestamp fallback only for manual verification, not auto-resolution.
   When root.log is unavailable, timestamp comparison can suggest whether a
   package likely has the fix, but cannot prove it due to:
   - Overlapping builds (buildroot updated mid-build)
   - Buildroot tagging delays
   - Build system caching of dependencies

root.log is preferred because it's precise and handles edge cases (cached
packages, epoch overrides). Timestamp fallback is needed because build logs
are not preserved indefinitely.

Performance: concurrent Koji lookups, HEAD checks before downloads, 30s
overall timeout across all architectures.

Security: validates package names, escapes JQL, validates NVR format.

Addresses: PACKIT-5421
Related: PACKIT-5289

Assisted-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Distinguish between 'CVE doesn't apply' and 'fix already shipped' scenarios:

1. CVE doesn't apply (NOT_AFFECTED):
   - Vulnerable code not present, version not affected, etc.
   - Routes to reproducer for verification
   - Gets ymir_triaged_not_affected label

2. Fix already shipped (ALREADY_FIXED):
   - Rebase/backport/rebuild already completed
   - Package has fix, needs errata processing
   - Routes to comment_in_jira, not reproducer
   - Gets ymir_triaged_already_fixed label

Changes:
- Added Resolution.ALREADY_FIXED enum and AlreadyFixedData model
- Updated triage prompt: use 'already-fixed' for completed rebases
- Added output format example with package_nvr and package_issue_key
- Excluded ALREADY_FIXED from reproducer handoff
- Added to _should_update_jira and comment_in_jira routing
- Included in _RESOLUTION_TO_LABEL mapping

For rebuilds: verify_rebuild_buildroot detects already-fixed via root.log
For rebases: triage detects version already at target via spec file check

Addresses code review feedback on PACKIT-5421

Assisted-by: Claude Sonnet 4.5 <noreply@anthropic.com>
…ifications

Three improvements to rebuild detection accuracy:

1. **Multi-architecture verification**: Fetch and verify all architecture
   root.logs agree on dependency version. Different archs can have different
   buildroots during long builds. Request clarification on conflicts.

2. **Active build handling**: Search both closed and active (not yet closed)
   builds. Trust closed builds immediately. Request clarification when only
   active builds exist, since they might be rejected before closure.

3. **Metadata failure handling**: Return clarification instead of False when
   Koji metadata unavailable. Missing metadata is indeterminate, not proof
   rebuild is needed.

New clarification reasons: architecture_dependency_conflict,
active_builds_not_closed, timestamp_comparison_failed.

Prevents false positives and false negatives by requesting human verification
for ambiguous cases.

Assisted-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Documents how Ymir determines whether a package needs to be rebuilt for
a CVE fix, including multi-architecture verification, active build
handling, and clarification scenarios.

Assisted-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Retry automatically on Jira/Koji outages instead of asking the user.
Fetch root.log only for architectures actually built (from Koji RPMs).
Skip noarch-only builds (no per-arch logs). Re-raise for recent builds,
fall back to timestamps for old ones past Brew retention.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 2f9fe55 to 560a4d3 Compare October 9, 2026 10:06
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

Comment thread ymir/common/utils.py
Comment thread ymir/tools/privileged/jira.py
Comment thread ymir/common/utils.py
Comment thread ymir/common/utils.py Outdated
Comment thread ymir/common/utils.py Outdated
@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 560a4d3

…logic, handle partial logs

- Fix noarch installed_pkgs.log location: fetch from noarch/ directory instead of trying builder architectures
- Remove date-based retry logic for missing logs: when logs are missing (404/410), don't retry - fall back to timestamp comparison immediately
- Fix partial log handling: return partial_architecture_coverage clarification when some (but not all) arch logs available, instead of discarding partial data and falling back to timestamp comparison
- Update documentation to reflect noarch/ directory and removal of date-based logic

- Use Koji API to resolve independently versioned subpackages
The issue: _parse_dependency_from_installed_pkgs_log constructs a source NVR as
{dep_component}-{version}-{release} using a subpackage's version. For packages
like perl where subpackages have independent versions (e.g., perl-Errno-1.30),
this creates non-existent Koji builds like perl-1.30, causing TransientInfrastructureError.

Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@majamassarini
majamassarini force-pushed the fix-packit-5421-rebuild-check branch from 560a4d3 to 510ddcc Compare October 9, 2026 12:23
@majamassarini

Copy link
Copy Markdown
Member Author

/agentic_review

@qodo-for-packit

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 510ddcc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants