Skip to content

fix_build_error: hint that builds run in COPR, not official builders - #644

Merged
opohorel merged 1 commit into
packit:mainfrom
opohorel:fix_build_prompt
Jul 1, 2026
Merged

opohorel merged 1 commit into
packit:mainfrom
opohorel:fix_build_prompt

Conversation

@opohorel

Copy link
Copy Markdown
Collaborator

The fix-build-error agent was modifying the spec file to work around COPR-specific environmental issues (unbuffer/expect crashes, pipefail behavior). Add context that builds run in COPR so the agent recognizes spurious environment failures and reports them instead of hacking the spec.

Assisted-by: Cursor (claude-4.6-opus)

The fix-build-error agent was modifying the spec file to work around
COPR-specific environmental issues (unbuffer/expect crashes, pipefail
behavior). Add context that builds run in COPR so the agent recognizes
spurious environment failures and reports them instead of hacking the spec.

Assisted-by: Cursor (claude-4.6-opus)

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the prompt_fix_build_error.j2 template to instruct the agent on how to handle build failures caused by COPR environmental differences rather than code changes. The reviewer suggested a helpful improvement to make the prompt instructions more precise by explicitly referencing the schema fields (success and error) instead of using generic phrasing.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread ymir/agents/prompts/backport/prompt_fix_build_error.j2
@qodo-for-packit

Copy link
Copy Markdown

PR Summary by Qodo

Hint fix-build-error agent that builds run in COPR (avoid spec hacks)

✨ Enhancement 📝 Documentation 🕐 Less than 5 minutes

Grey Divider

AI Description

• Clarify that fix-build-error builds execute in COPR, not official RHEL builders.
• Instruct the agent to treat COPR-specific failures as environmental and report them.
• Prevent spec-file workarounds for spurious COPR issues (e.g., expect/pipefail/locale).
Diagram

graph TD
  A["fix-build-error agent"] --> B["Prompt: fix_build_error.j2"] --> C{{"COPR env variance"}} --> D["Report success=false + explain"]
  B --> E["Avoid spec modifications"]

  subgraph Legend
    direction LR
    _agent["Agent"] ~~~ _doc["Prompt / policy"] ~~~ _dec{{"Environment cause?"}} ~~~ _out["Outcome"]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Add explicit env-failure classification in agent code
  • ➕ More reliable than prompt-only steering; less dependent on LLM interpretation
  • ➕ Can standardize known COPR failure signatures (e.g., expect/unbuffer crashes)
  • ➕ Easier to test/validate with unit tests or fixtures
  • ➖ Higher implementation cost; requires code changes and release/rollout
  • ➖ Risk of overfitting to current COPR quirks; needs ongoing maintenance
2. Improve CI/build logs with COPR fingerprinting and guidance
  • ➕ Makes the environment provenance unambiguous in logs
  • ➕ Helps both humans and agents quickly triage environmental vs patch failures
  • ➖ Requires pipeline/logging changes outside the agent prompt
  • ➖ Still may need agent-side policy to avoid spec workarounds

Recommendation: Prompt-level guidance is a low-cost, high-leverage fix for the immediate failure mode (agent hacking the spec for COPR quirks). If COPR spurious failures continue, follow up with lightweight agent-code heuristics to detect and classify common COPR-environment signatures more deterministically.

Files changed (1) +5 / -0

Documentation (1) +5 / -0
prompt_fix_build_error.j2Add COPR-environment constraint to prevent spec-file workarounds +5/-0

Add COPR-environment constraint to prevent spec-file workarounds

• Extends the critical constraints to clarify builds run in COPR rather than official RHEL builders. Instructs the agent to treat COPR-specific failures as environmental (report success=false with explanation) and explicitly avoid modifying the spec file to work around those issues.

ymir/agents/prompts/backport/prompt_fix_build_error.j2

@qodo-for-packit

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Context used
✅ Compliance rules (platform): 7 rules

Grey Divider


Remediation recommended

1. Non-actionable failure retries 🐞 Bug ➹ Performance
Description
The prompt now instructs the agent to return success=false for COPR environment-caused failures, but
the workflow retries fix_build_error on any success=false up to max_incremental_fix_attempts. For
persistent environmental failures this causes repeated, non-actionable LLM runs before the workflow
finally exits.
Code

ymir/agents/prompts/backport/prompt_fix_build_error.j2[R28-32]

+  The build runs in COPR, not on official RHEL builders. COPR environments may have
+  differences (e.g. unbuffer/expect wrappers, pipefail behavior, locale settings) that
+  can cause spurious failures unrelated to your patches. If the failure is caused by the
+  build environment rather than by your code changes, report success=false and explain
+  the environmental issue — do not modify the spec to work around it.
Relevance

⭐⭐⭐ High

Team accepted retry-loop fixes to avoid pointless repeats and improve failure semantics (PRs 540,
273, 616).

PR-#540
PR-#273
PR-#616

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The prompt explicitly tells the agent to return success=false for COPR environment failures, while
the backport workflow retries fix_build_error whenever the agent returns success=false and attempts
remain, making environmental failures non-actionable but still retried.

ymir/agents/prompts/backport/prompt_fix_build_error.j2[21-34]
ymir/agents/backport_agent.py[450-532]
ymir/agents/backport_agent.py[578-588]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The fix-build-error prompt instructs the agent to return `success=false` for COPR environmental failures, but `fix_build_error` treats all `success=false` results as retryable and will re-run the agent repeatedly (bounded by `max_incremental_fix_attempts`). This wastes attempts when the failure is explicitly non-actionable.

## Issue Context
- The new prompt guidance is correct (don’t hack the spec for COPR quirks), but the workflow needs a way to recognize “environmental / non-actionable” failures and terminate early (e.g., comment in Jira immediately) rather than retrying.

## Fix Focus Areas
- ymir/agents/prompts/backport/prompt_fix_build_error.j2[21-34]
- ymir/agents/backport_agent.py[450-532]
- ymir/agents/backport_agent.py[540-589]

## Suggested approach
1. Introduce a structured/recognizable signal for environmental failures (preferred: add a boolean field like `environment_failure` to `BackportOutputSchema`; fallback: require a stable error prefix like `COPR_ENVIRONMENT_FAILURE:`).
2. In `fix_build_error`, if that signal/prefix is present, skip retries and transition directly to `comment_in_jira` with the original diagnosis.
3. Update the prompt to instruct the agent to set the new field (or to use the required prefix) when it determines the failure is COPR-environment-related.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@TomasTomecek TomasTomecek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice improvement

@lbarcziova lbarcziova left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@opohorel
opohorel merged commit 219aa7c into packit:main Jul 1, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants