fix(harness): sandbox edit generated python3 -c program with literal sequences (SyntaxError) by aiyili · Pull Request #3226 · agentscope-ai/agentscope-java

aiyili · 2026-09-21T05:32:49Z

Problem

BaseSandboxFilesystem.edit(...) failed before reading or modifying the target file. Two defects in the generated shell command:

Literal sequences instead of line feeds. The Java code joined the python3 -c "..." program fragments with \n (a literal backslash + n at runtime). Inside a POSIX double-quoted string the backslash before n is not an escape — the two characters reach python3 unchanged — so the whole program arrived as a single line and the interpreter failed with:
```
SyntaxError: unexpected character after line continuation character
```
Exit code ignored. execute's response was parsed without checking the exit code, so the interpreter crash above was masked as unexpected server response instead of the actual cause.

Fix

Build the Python program with a Java text block, so its statements are separated by real line feeds. The statements themselves are unchanged. A comment documents why real line feeds are required (regression guard).
Check result.isSuccess() right after execute, reusing the existing executeFailureMessage(...) convention from ls/read/grep/glob: execution-layer failures (timeout 124, transport errors, interpreter crash) now surface with their diagnostic output (or exit code fallback) instead of being mislabeled.

Verification

Reproduced against a real sh + python3 locally: the old command fails with the SyntaxError above and leaves the file untouched; the fixed command outputs {"count": 1} and performs the replacement.
agentscope-harness module test suite: BUILD SUCCESS; BaseSandboxFilesystemTest 19/19 canned tests pass (incl. 3 new ones), 3 new Linux real-shell integration tests run on CI.
mvn spotless:check passes.

Tests added

Unit (all platforms):

edit_generatedPythonProgram_isSeparatedByRealLineFeeds — the generated command contains real line feeds and no literal sequence
edit_executeFailure_shouldFailWithCause — transport failure surfaces with its cause
edit_executeFailure_nullOutput_shouldFailWithExitCodeFallback — exit-code fallback message

Linux real-shell integration:

edit_singleOccurrence_replacesIt — replaceAll=false on a single occurrence replaces it (the scenario from the issue)
edit_multipleOccurrencesWithoutReplaceAll_failsAndKeepsFileUnchanged
edit_replaceAll_replacesEveryOccurrence

CLAassistant · 2026-09-21T05:32:56Z

All committers have signed the CLA.

oss-maintainer

Summary

Well-diagnosed fix. Switching the embedded python3 -c program to a Java text block makes the line feeds real (a literal \\n inside POSIX double quotes reaches python3 unchanged and collapses the program into one line), and the added isSuccess() guard aligns edit() with the existing executeFailureMessage(...) convention used by ls/read/grep/glob. CLA signed; unit + Linux real-shell regression tests are a good addition. LGTM.

One note: this PR currently shows mergeStateStatus=BLOCKED on GitHub — likely pending CI approval/reviews, not a code issue.

Automated review by github-manager-bot

codecov · 2026-09-21T05:53:54Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

oss-maintainer · 2026-09-21T20:37:35Z

CLA Not Signed

The Contributor License Agreement (CLA) check is currently pending on this PR (license/cla: Contributor License Agreement is not signed yet.). This PR cannot be merged until the CLA is signed.

@aiyili please sign the CLA via the CLA assistant badge in the comment above, or visit https://cla-assistant.io/agentscope-ai/agentscope-java. Once signed, the license/cla status will turn green.

Automated check by github-manager-bot

aiyili · 2026-09-21T23:11:50Z

@cla-assistant check

… SyntaxError - 改用 Java 文本块构建 python3 -c 程序,语句间为真实换行符:POSIX 双引号字符串内字面反斜杠+n 序列会原样传给 python3,整个程序被压成一行直接 SyntaxError - execute 返回后先检查退出码:执行层失败(超时/传输错误/python 崩溃)现在携带原始原因返回,不再被掩盖成 unexpected server response - 新增单元测试(生成命令形态 + 失败响应)与 Linux 真实 shell 集成测试 (单次替换 / 多处出现拒绝 / replaceAll 全替换) Fixes agentscope-ai#2571

aiyili · 2026-09-22T05:30:08Z

Cross-referencing the #2571 evaluation's two reviewer asks against this PR:

1. The extra replaceAll=true finding — covered by construction: the fix rewrites the command builder once (text block with real line feeds), and replace_all remains pure payload data, so both edit_file branches share the corrected program. edit_replaceAll_replacesEveryOccurrence exercises the true branch end-to-end on a real shell (Linux CI).

2. Pinning the command construction — edit_generatedPythonProgram_isSeparatedByRealLineFeeds (runs on all platforms) asserts the generated command starts with python3 -c ", is separated by real line feeds, and contains no literal backslash-n sequence anywhere; the base64 heredoc payload is asserted intact by the three Linux real-shell integration tests (edit_singleOccurrence_replacesIt covers the exact issue scenario, plus multiple-occurrence refusal and replaceAll).

Additionally, this PR also fixes the second symptom from the report — the exit code is checked before parsing, so execution-layer failures (timeout / transport / interpreter crash) surface with their cause instead of being masked as unexpected server response (edit_executeFailure_shouldFailWithCause, edit_executeFailure_nullOutput_shouldFailWithExitCodeFallback).

Verified end-to-end against a real sh + python3 before submission (old command: SyntaxError, file untouched; fixed command: replacement applied). The branch is rebased onto current main and clean.

gmjneko · 2026-09-22T15:04:29Z

LGTM. But duplicates #2573

Resolves the conflict introduced by agentscope-ai#3226, which rewrote the same edit() method on the same class. Our implementation supersedes it, so ours is kept; agentscope-ai#3226's substance is preserved as follows. - agentscope-ai#3226's three canned tests are carried over. The one asserting python3 -c was retargeted at the heredoc form: the real invariant behind agentscope-ai#2571 is that program statements are separated by real line feeds, not that the two-character sequence backslash-n never appears (the CRLF normalizer legitimately uses it inside Python string literals). Its two executeFailure tests now pass because edit() reports execution-layer failures through the same executeFailureMessage() convention as ls/read/grep/glob, which edit() was the last method not to use. - agentscope-ai#3226's three integration tests are behaviourally identical to the edit_simpleReplacement / edit_multipleOccurrencesWithoutReplaceAll / edit_replaceAll tests already here; the one assertion they added (a rejected ambiguous edit leaves the file untouched) is folded into ours. Also closes the remaining half of the CRLF parity finding: the native path performed byte-exact replacement while the transfer fallback normalized, so the same edit could succeed or fail depending on whether python3 was present. The script now applies the same LF normalization as FilesystemUtils.stringReplacement. The escapes in that snippet are doubled because a Java text block would otherwise emit real CR/LF bytes and split the Python string literal across lines.

… output (agentscope-ai#3262) On a Windows host the argv -> command-line -> docker.exe re-parse round-trip drops inner double quotes; a double-quoted span containing a space then breaks into argv separators and the in-container shell fails with a syntax error, so write_file always failed there. - write(): mkdir -p "$(dirname <p>)" -> d=$(dirname <p>); mkdir -p "$d" (same quote-free-spread pattern move() already uses); no double-quoted span contains a space anymore, no POSIX behavior change - write(): the pre-check failure branch now appends checkResult.output() (exit-code fallback when blank) instead of returning a bare reason-less 'Failed to file' message, matching the upload branch's convention edit() deliberately left untouched: its python3 -c rewrite belongs to in-flight agentscope-ai#3084 / agentscope-ai#3226. Regression tests: command contains no double-quoted span with a space; failure message carries the container's error output; exit-code fallback. Sabotage-verified: restoring the old construction fails the new test.

agentscope-ai#3226 rewrote edit() with a text-block program plus an exit-code check, so this branch now takes main's version of edit() and drops the base64-environment payload approach that overlapped it. Remaining scope: write()/move() parent-dir quoting and write()'s failure message.

oss-maintainer approved these changes Sep 21, 2026

View reviewed changes

oss-maintainer mentioned this pull request Sep 21, 2026

[Bug]: BaseSandboxFilesystem.edit fails when replaceAll=false because the generated Python command contains literal \\n sequences` #2571

Closed

aiyili force-pushed the main branch from 7867333 to 8adb544 Compare September 21, 2026 23:14

This was referenced Sep 23, 2026

[Bug] Windows 宿主上 write_file / edit_file 必然失败：命令中的双引号在传参往返中被丢弃 #3262

Open

fix(harness): remove quoted-space write command and surface its error output #3269

Open

jujn assigned gmjneko Sep 28, 2026

jujn merged commit 5257062 into agentscope-ai:main Sep 28, 2026
6 checks passed

jujn mentioned this pull request Sep 28, 2026

fix(harness): use real newlines in sandbox edit python -c #2573

Closed

5 tasks done

chcodex mentioned this pull request Sep 28, 2026

fix(harness): run native inline-python edit in sandbox with transfer fallback #2398

Merged

gh-zhaoweifeng mentioned this pull request Sep 29, 2026

fix(sandbox): use base64+stdin for edit() to avoid bash -lc escaping … #3084

Closed

5 tasks

aiyili mentioned this pull request Sep 30, 2026

fix(harness): contain CompositeFilesystem root ls/grep to the backend root (stacked on #3247) #3254

Open

King52HerTz mentioned this pull request Oct 11, 2026

fix(harness): build sandbox commands without load-bearing quotes #3329

Open

aiyili mentioned this pull request Oct 21, 2026

fix(core): stamp id/name on the Toolkit.callTool single-call path #3686

Closed

Navigation Menu

Repository navigation

fix(harness): sandbox edit generated python3 -c program with literal sequences (SyntaxError) - #3226

fix(harness): sandbox edit generated python3 -c program with literal sequences (SyntaxError)#3226
jujn merged 1 commit into
agentscope-ai:mainfrom
aiyili:main

aiyili commented Sep 21, 2026

Uh oh!

CLAassistant commented Sep 21, 2026 •

edited

Loading

Uh oh!

oss-maintainer left a comment

Uh oh!

codecov Bot commented Sep 21, 2026

Uh oh!

oss-maintainer commented Sep 21, 2026

Uh oh!

aiyili commented Sep 21, 2026

Uh oh!

aiyili commented Sep 22, 2026

Uh oh!

gmjneko commented Sep 22, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

aiyili commented Sep 21, 2026

Problem

Fix

Verification

Tests added

Uh oh!

CLAassistant commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oss-maintainer left a comment

Choose a reason for hiding this comment

Summary

Uh oh!

codecov Bot commented Sep 21, 2026

Codecov Report

Uh oh!

oss-maintainer commented Sep 21, 2026

Uh oh!

aiyili commented Sep 21, 2026

Uh oh!

aiyili commented Sep 22, 2026

Uh oh!

gmjneko commented Sep 22, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CLAassistant commented Sep 21, 2026 •

edited

Loading