Skip to content

Improve Algorithm and Experiment agent workflows - #5

Open
HuangZhengyang-2004 wants to merge 5 commits into
mainfrom
zane
Open

HuangZhengyang-2004 wants to merge 5 commits into
mainfrom
zane

Conversation

@HuangZhengyang-2004

@HuangZhengyang-2004 HuangZhengyang-2004 commented Aug 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • integrate traceable ReaScholar evidence retrieval into the Algorithm and Experiment workflows
  • refine the two agent prompts and supporting skills while keeping model selection in the global runtime configuration
  • harden the Experiment execution contract so missing entrypoints and vacuous execution fields cannot be marked ready
  • temporarily disable smart-plotting mounts and trigger prompts for evaluation while leaving the plotting scripts unchanged
  • keep codex-config.toml, Intro, Survey, and Prover unchanged from main

Testing

  • Algorithm/Experiment scope: 18 tests passed
  • Full suite: 40 passed, 1 failed
    • the remaining failure is test_tex_inputs_are_expanded_and_citations_are_preserved
    • the same Intro path failure reproduces on origin/main; this PR intentionally does not modify Intro
  • git diff --check passed
  • independent pre-PR review found no remaining Critical or Important issues

CPU30 16-paper ReasFlow / Codex Native experiment archive

Adds the complete saved experiment evidence for the 16 matched cpu30-ab-v3 papers (run 003): 32 runs covering Codex Native and ReasFlow Experiment Agent. The repository includes a browsable per-paper index, original Midloop/Bigloop judge results, analysis, and later algorithm-judge reports. Original scoring and subsequent numerical checks remain separate; this archival change does not rerun or alter scores.

  • Experiment index
  • Complete evidence attachments: workspaces, results, plots/data, prompts/configuration, raw session traces, logs, runtime records, judge source, all saved judge attempts/reviews/controls, and file manifests.
  • Preserves the local evidence snapshot and the full original run directories from both source servers separately. Later follow-up evidence is explicitly a snapshot, not a claim of completed follow-up evaluation.

Validation: all 34,139 transferred source files matched server SHA-256; all 32 main traces matched the historical trace index; 43,120 archived files were verified against the manifest, including losslessly deduplicated files. The original judge directories are present for all 32 runs. Credentials are excluded/redacted, and reinstallable environments/caches and system symlinks are omitted with records in the archive. SQLite credential redaction includes removal of stale bytes. Bulk evidence is marked as generated for review while its original whitespace is preserved.

This update stays on zane and in this open PR; the default branch is not merged or changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant