Skip to content

Gate HRX pin updates on long-prompt validation - #379

Merged
bong-water-water-bong merged 3 commits into
mainfrom
copilot/fix-end-of-text-issue
Oct 9, 2026
Merged

bong-water-water-bong merged 3 commits into
mainfrom
copilot/fix-end-of-text-issue

Conversation

Copilot AI commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Short-prompt and backend-op checks missed an HRX regression that returned end-of-text on a 4.7K-token Qwen3-0.6B prompt. Require pin-specific long-prompt validation before an HRX submodule update is accepted.

  • Probe: Add tests/hrx_long_prompt.py to warm up llama-server, confirm the prompt is at least 4,700 tokens, and require the buried-code answer 7341.
  • Pin gate: CI rejects a changed HRX pin unless its full commit matches config/hrx-long-prompt-validated-pin.config. Update that record only after validating the proposed pin on Strix Halo:
    python3 tests/hrx_long_prompt.py
  • Documentation and coverage: Describe the validation procedure and add focused tests for the pin gate and probe prompt.

Copilot AI and others added 2 commits October 9, 2026 18:52
Co-authored-by: bong-water-water-bong <277547417+bong-water-water-bong@users.noreply.github.com>
Co-authored-by: bong-water-water-bong <277547417+bong-water-water-bong@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix Qwen3-0.6B returning end-of-text on 4.7K-token prompt Gate HRX pin updates on long-prompt validation Oct 9, 2026
@bong-water-water-bong
bong-water-water-bong marked this pull request as ready for review October 9, 2026 19:28
@bong-water-water-bong
bong-water-water-bong enabled auto-merge (squash) October 9, 2026 19:28
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅

369 - PR Code Verified

Compliant requirements:

  • Fix the regression where Qwen3-0.6B returns end-of-text at once on a 4.7K-token prompt with HRX backend
  • Gate HRX pin updates on long-prompt validation
  • Add a probe test to verify correct behavior on long prompts
  • Update documentation to describe the validation procedure

Requires further human verification:

  • Verification that the proposed hrx-system pin passes the long-prompt test on Strix Halo
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Potential Timeout Issues

The hrx_long_prompt.py script uses a 300-second timeout for HTTP requests. In the case of a regression or performance degradation, this timeout may not be sufficient to complete the long-prompt test, potentially masking issues. Consider making the timeout configurable or increasing it to ensure robustness.

with urllib.request.urlopen(request, timeout=300) as response:
    return json.load(response)
Validation Error Message Clarity

The error message in hrx_validation_error function could be more specific about what constitutes a valid commit. While it mentions running tests/hrx_long_prompt.py, it doesn't clearly state that the commit must be validated on Strix Halo before being accepted.

def hrx_validation_error(path, new, validated):
    if path == HRX_PATH and new != validated:
        return (f"hrx-system pin {new[:12]} has not passed the long-prompt check; "
                f"run tests/hrx_long_prompt.py on Strix Halo and update {HRX_VALIDATED_PIN}")
    return None

@bong-water-water-bong
bong-water-water-bong merged commit 8d4d0fa into main Oct 9, 2026
7 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the copilot/fix-end-of-text-issue branch October 9, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

hrx-system 4ba76c18eafe: Qwen3-0.6B returns end-of-text at once on a 4.7K-token prompt (98d05d94 answers)

2 participants