Skip to content

feat: image archive — folded ranges keep re-readable image pointers instead of dropping pixels (#478) - #479

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-15_image-archive
Oct 15, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-15_image-archive

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

Images attached as data-URL file parts were silently dropped when their range was folded: the summary carried no record of them and the pixels were unrecoverable once OpenCode storage lost the originals. This ports the billion-context #1995/#2640 principle per issue #478: images are not summarized — they are dumped to a file that the model re-reads with its own tools.

How

  • Eager spill (lib/compress/image-archive.ts, new): a per-turn pass after assignMessageRefs decodes qualifying images (data: URL, binary mime png/jpeg/gif/webp/avif/bmp — SVG excluded as scriptable text, size ≥ minBytes) into <storageDir>/images/<dirName(sessionId)>/ as mode-0600 files named <sha8(rawMsgId)>-p<i>-<sha8(content)>.<ext> (content-salted so skip-if-exists is safe across ref re-issue). A metadata-only index (path/mediaType/bytes, keyed by raw message ID) persists on SessionState.imageArchive; it survives compaction ref re-issue without clearing.
  • Pointer appendix at fold time (lib/compress/range.ts): last step of the summary chain appends - <ref>: <abs path> (<mime>, <bytes>) lines for archived images covered by the folded range. No new tool, no system-prompt surface — the model re-reads pixels with its existing file tools.
  • TTL sweep: best-effort prune of files older than ttlDays, once per process per session, never blocking a request. All archive I/O failures are warn-logged, never thrown.

Behavior changes (disclosed)

Area Old New
Folded range containing archived image image bytes lost, no trace in summary summary gains pointer appendix with absolute paths; bytes on disk re-readable
Disk usage none from ACP for images <storageDir>/images/<dirName(sid)>/ (0600), TTL-pruned after 7 days default
Session state file no image fields optional imageArchive index + optional stats.imagesArchived/imageBytesArchived; old state files load unchanged
Config — new compress.images { enabled=true, minBytes=16384, ttlDays=7 }, top-level compress node only (excluded from per-model override cascade — archive location is session-scoped)

enabled: false restores pre-change behavior exactly (no disk writes, no appendix). No changes to internal dcp-* tags, block schema, or prompt-cache strategy.

Tests

  • New tests/image-archive.test.ts: 8 unit tests (parse/ext/dir-name helpers; spill permissions/dedup/skip matrix; TTL prune semantics incl. once-per-state guard; pointer rendering incl. ref resolution).
  • Targeted regression: compress-range + config-validation + compress-state + e2e transform/block suites → 105/105 pass locally. Full suite delegated to CI.
  • Typecheck + build + prettier clean; scripts/ci/check-pr.sh passes.

Closes #478

…read pointers into folded summaries (#478)

Images attached as data-URL file parts were silently dropped when their
range was folded, with no recovery path once OpenCode storage lost the
originals. Port the billion-context #1995/#2640 principle:

- archiveSessionImages(): per-turn pass after assignMessageRefs spills
  qualifying images (data URL, binary mime, >= minBytes) to
  <storageDir>/images/<dirName(sessionId)>/ as 0600 files named
  <sha8(rawId)>-p<i>-<sha8(content)>.<ext>; metadata index persisted on
  SessionState.imageArchive keyed by raw message ID (survives ref
  re-issue); never throws.
- appendArchivedImagePointers(): last step of the range-mode summary chain
  appends absolute-path pointer lines for archived images covered by the
  folded range; model re-reads pixels with its own file tools.
- pruneImageArchive(): best-effort TTL sweep once per process per session.
- Config: compress.images { enabled=true, minBytes=16384, ttlDays=7 },
  excluded from the per-model override cascade; disabled = exact status
  quo. Schema + validation + docs updated.

Closes #478
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Plugin Artifact

Branch: 2026-10-15_image-archive (e5ea705)

Option A — Install from npm PR tag (recommended)

opencode plugin opencode-acp@pr-479 --global

Each push to this PR publishes a new version under the pr-479 npm tag.

Option B — Install from GitHub

opencode plugin "github:ranxianglei/opencode-acp#2026-10-15_image-archive" --global

Option C — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf opencode-acp-pr479.tgz
cp -r package/dist ~/.cache/opencode/packages/opencode-acp@latest/node_modules/opencode-acp/dist
  1. Restart opencode to pick up changes.

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 查重:不重复——全仓 issue/PR 检索「image / 图片 / archive / attachment」仅命中本 PR(#479)与其源需求 #478,无其他图片处理相关条目。

开始独立审核(不复用 #478 工作会话的结论):拉取分支 2026-10-15_image-archive @ b001023,逐文件审 diff、定向跑测试、核对 CI。当前 CI:pr-validation ✅、build-artifact ✅、test(22/24) 运行中。结论稍后按结构化八问给出。

Review hardening for the image-archive feature: the documented contract
('excluded from the per-model override cascade') only held at the type
level. resolveCompressOverrides spreads whatever user JSON sits under
compress.providers/<p> or .models/<m>, so an images entry there reached
applyCompressOverrides and was blanket-applied as a PARTIAL object —
replacing the global images config wholesale (minBytes undefined disables
the size filter, ttlDays undefined makes the TTL cutoff NaN).

Add 'images' to OVERRIDE_BLANKET_APPLY_EXCLUDE (widening its element type,
since CompressModelOverrides Omits it) plus a regression test covering both
the runtime no-leak behavior and the existing validation rejection of the
misplacement. No behavior change for valid configs.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review 完成。结论:可合入——审核中发现一个真实缺陷(「images 排除在级联之外」只在类型层生效,运行时可被绕过),已直接推修复 commit 到 PR 分支(e5ea7055),对应 CI 正在收尾。合并操作请人工执行。

结构化八问

  1. 是什么问题:含图片附件的 range 被折叠后,图片字节被静默丢弃、summary 里没有任何痕迹——OpenCode 自身存储一旦丢失原件即不可恢复。本 PR 移植 billion-context #1995/#2640 的原则:图片不做摘要,落盘成文件,模型需要时用自己的工具重读。

  2. 什么原因导致的 / 怎么修的:根因是 FilePart 只活在原始消息里,折叠后随消息消失、无其他引用。修法三段式:①每轮 transform 在 assignMessageRefs 之后立即把达标图片(data: URL、二进制类型 png/jpeg/gif/webp/avif/bmp、≥minBytes=16KB)以 0600 权限写到 <storageDir>/images/<dirName(sessionId)>/<sha8(rawId)>-p<i>-<sha8(content)>.<ext>;②元数据索引(keyed by raw message ID,compaction 重发 ref 不受影响)持久化到 SessionState.imageArchive;③折叠时 summary 链最后一步追加指针行 - mNNNNN: /abs/path (mime, N bytes),模型用既有文件工具重读。无新工具、无 system prompt 面。TTL 清扫 best-effort、每进程每会话一次;全部磁盘 I/O 失败仅 warn 日志、绝不中断请求。

  3. 影响范围与严重程度:纯增量功能,默认开启。影响面 = 带达标图片附件的会话:新增磁盘写入(TTL 默认 7 天)、覆盖这些图片的 summary 变长(每条指针行几十字节)。enabled:false 或无达标图片时行为逐字节不变(两条路径都验证过提前返回)。对存量用户无行为改变,最坏情况是磁盘占用增长,有 TTL 兜底。

  4. 怎么修复的(review 侧动作):独立逐项审核(未复用 支持图片压缩,参考billion-context的对图片支持原理 #478 线程结论),并推了一个加固 commit e5ea7055:「images 排除在 per-model override 级联之外」此前只有类型层的 Omit——resolveCompressOverrides 会把用户在 providers/<p> 或 .models/<m> 下写的任意字段整体展开,applyCompressOverrides 再 blanket-apply,于是误放位置的 images 会以部分对象整体替换全局值(minBytes undefined → 尺寸过滤失效;ttlDays undefined → TTL 阈值 NaN → prune 永不触发)。修复:images 加入 OVERRIDE_BLANKET_APPLY_EXCLUDE(lib/messages/inject/utils.ts)+ 回归测试(已验证去掉修复则测试红、加上则绿)。

  5. 是否有回归问题:审过的既有行为均无回归——pipeline 插入点(assignMessageRefs 之后、syncCompressionBlocks 之前,只新增状态字段,不触碰 ref/block/nudge 逻辑);config 合并(旧配置加载不变,新增字段全 optional,VALID_CONFIG_KEYS 与类型校验规则正确);状态持久化(旧 state 文件原样加载 + 畸形索引条目防御性过滤);dcp-* 标签、block schema、prompt-cache 策略零改动;高危模式逐项过:无 fail-fast 变吞错(archive 失败 best-effort 是 DESIGN.md 明确的设计决策)、无阈值/时序/输出格式漂移(appendix 只出现在新生成的 summary 里)。定向本地测试 5 个相关套件 102/102 通过、typecheck 干净;CI 全套:修复前 5/5 run 全绿,修复后 pr-validation/build 已绿,test 矩阵运行中。
    我的修复 commit 引入的行为变更(披露):仅限无效配置场景——把 images 放在 providers/models 下的用户:旧行为 = 部分对象替换全局(尺寸过滤失效 + TTL 失效);新行为 = 全局 images 正常生效,且 validation 照旧报 unknown field 错误指向该键。合法配置完全不受影响。

  6. 是否破坏缓存:否。prompt-cache 策略不变;archive 工作发生在 wire 之外;appendix 只进入新建 block 的 summary(其本身就是 cache-churn 事件);索引 keyed by raw ID,compaction 不影响。

  7. 是否改了设计方向:没有转向——沿用 BC 已验证的 file-first 恢复机制,工具/prompt 面零新增。四个已知限制(设计层面,列出供 owner 知悉,均不阻塞合入):①TTL 清扫只覆盖当前会话目录且每进程生命周期一次 → 死会话目录要等该会话在某进程再次活跃才清理(慢速磁盘增长尾部);②源消息仍在时,过期删除的文件会被下一轮重写(指针自愈,语义可接受);③T2/T3 distillation 可能丢掉指针行(与 BC 一致,v1 可接受);④每轮对每个未折叠图片做一次 sha8 全内容哈希(hot path 上 O(payload) CPU,性能观察项)。

  8. 是否引入了新的配置:是——compress.images { enabled=true, minBytes=16384, ttlDays=7 },仅顶层 compress 节点,排除在 per-model 级联外(存档位置必须会话内稳定)。无需迁移,旧配置自动取默认值。

其他审核记录

  • 测试有效性(§5.6):import 均来自真实源模块;断言有牙(0o600 权限位、dedup no-op、utimesSync 真实删文件、once-per-state guard、ref 解析 + raw fallback、disabled→逐字节不变);无恒真断言。两个小缺口(不阻塞):statSync 恢复路径(文件在、索引丢)无专门测试;多 part 的 p<i> 命名未直接断言。
  • Devlog:REQ/WORKLOG/DESIGN 齐备;DESIGN 记录了各决策取舍(eager vs lazy 落盘、raw-ID 索引、SVG 排除理由、once-per-process TTL、级联排除)。
  • 小观察(不阻塞):作者提交的 tests/config-providers.test.ts 里约 57 行不符合仓库钉死的 prettier 3.9.5(.prettierrc)格式——我新增的部分已格式化,未动那些行;CI 不以 prettier 为门禁,不影响合并。

判定:可合入。 e5ea7055 的 test 矩阵跑绿后,请人工点击合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 补充确认:e5ea7055 的全部 CI(pr-validation / build / test Node 22+24)已跑完且全绿,PR 状态 clean。可以人工合并了:#479

@ranxianglei
ranxianglei merged commit ac1068a into master Oct 15, 2026
6 checks passed
ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
Completes the AGENTS.md 5.4.2 artifacts the one-click workflow's fallback PR
does not generate: CHANGELOG.md / CHANGELOG.zh-CN.md entries and
devlog/2026-10-15_release-v1.18.4/{REQ,WORKLOG}.md for the image-archive
feature shipped since v1.18.3. Version bump itself already landed via the
workflow's Bump step.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

支持图片压缩,参考billion-context的对图片支持原理

1 participant