Skip to content

[PR] feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生(默认关闭) - #5236

Merged
ranxianglei merged 14 commits into
masterfrom
feat-4313-message-timestamps
Oct 31, 2026
Merged

ranxianglei merged 14 commits into
masterfrom
feat-4313-message-timestamps

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

Resolves #4313(owner 2026-10-30 拍板实现)。

默认关闭(opt-in):本特性由 compress.messageTimestamps 开关控制(配置顶层 compress 块布尔值,或环境变量 BILI_MESSAGE_TIMESTAMPS;env 优先:0=强制关,其他任意值=开;未设置看配置文件,都没有=关)。关闭时线上出站字节与升级前逐字节一致。开启需重启 bili 实例。存储层无条件记录每条消息时间戳,所以先运行积累的历史在开启后完整浮现。


这是啥问题?

模型在线上看得到每条消息的 mNNNNN 标号,但看不到任何时间。两次实战后果:

  • 压缩块摘要只引用标号,模型无法判断"这块内容发生在几小时前还是几分钟前",把旧结论当现状用;
  • 长会话多次折叠后,模型分不清哪段是用户最近的指令、哪段是早已过期的旧指令(2026-10-30 两个真实会话都出现过:旧指令残留压过最新裁决,以及把折叠前已完成的旧任务当成待办重做)。

怎么解决?

每条消息盖一次不可变时间戳,渲染进现有 <acp> 标签;压缩块指纹行自动追加覆盖时间范围。

  • 标签变为 <acp tokens="2.1K" type="tool:bash" time="2026-10-30 09:02:11">m00175</acp>(完整 yyyy-mm-dd hh:mm:ss 本地时间,约 +8 token/条)
  • 缓存安全:时间戳在消息首次到达时铸一次、存进账本与统一存储,之后永远渲染同一个字节值——渲染路径不读墙钟,同快照两次渲染逐字节相同(有钉子测试)
  • 老消息零扰动:存量行不回填、保持 NULL,标签不带 time 属性,字节与升级前完全一致;fork/继承直接沿用旧行的旧时间
  • 提示词三处更新:① 标号小 = 早发生 ② 时间小 = 早发生 ③ 写压缩摘要时开头一行日记式 "Occurred: <起> – <止>";压缩块指纹行自动带 [起 – 止] 范围(覆盖区间任一消息无时间则整体回退旧格式,绝不谎报范围)
  • 统一存储 v10:messages 加可空 ts 列(sqlite+mysql),ts 走独立列不进 content 字节 → 内容 hash/身份完全不变
  • verif 子语言守卫按 createdAt 先例登记两条例外并注释理由

测试依据

  • kernel 全量套件 1495 pass / 0 fail(含新增 issue4313 回归套件:铸戳一次不复铸、渲染确定性、老消息原字节、跨块合并继承)
  • host 定向批(storage-unified / snapshot-shadow 字节级一致 / golden-storage-render 等)全绿;4 件 storage-render golden 按官方脚本再生成(掩码扩展与既有 port/sid/tz 同类,生产 canonicalize 未动)
  • codex e2e:t1–t6 全绿,t7 因当时本机模型服务挂死超时(独立直连探针确认环境故障,非代码回归)——CI lane 会覆盖复跑

一个一次性代价

升级后新消息首次渲染会带上 time 属性(前缀与旧字节不同 → 该会话当轮缓存自然重建一次);之后的请求全部命中稳定值。存量历史消息不受影响。


kernel-change: #4313 消息时间戳特性 — kernel/src/{compress-tools,compress,index,packs,render-refs,sync,types}.ts 新增不可变 ts 列铸造/渲染/跨块继承 + kernel/tests/*(issue4313 回归套件);kernel/package.json 0.0.171→0.0.173(基线为合并当前 master 后;master 已经由 #5253 占用 0.0.172,按两 PR 约定顺序规则顺延一号),lock 已同步再生成。owner 2026-10-30 拍板(#4313)。2026-10-31 已合并 master 解决冲突(#5188 sidecar 恢复以 master owner 版为准,本 PR 仅保留 #4313 改动),golden storage-render 按官方脚本重新生成。 2026-10-31 再次同步 master(其后新增的 #5253/#5292/#5294/#5163 等合并):零冲突,kernel 1507/1507 + host 存储/compress 定向批全绿,随本次推送。

…tsStamps mint, <acp> time attribute rendering (no wall-clock read on the render path), fold-receipt time spans, prompt ordering rules (smaller ref/time = earlier)
…unified schema v10 adds nullable messages.ts (sqlite+mysql, probe-first ALTER, ingest-stamp-once, NO backfill: pre-feature rows render byte-identical without the attribute), hydrate onto CoreMessage.ts for snapshot render/incremental lane/shadow reads, and append covered [start – end] span to compression summary fingerprint lines when resolvable
…print spans; storage v10 migration/watermark/hydration assertions; regenerate storage-render goldens (tsStamps/ts column in state digest) with wall-clock display normalization in the golden drive harness alongside the existing port/sid/tz masking
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: feat-4313-message-timestamps (1fa1f11)

Latest build — install this exact version

npm install -g billion-context@0.0.7384-pr.5236.160e879

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
1fa1f11 billion-context@0.0.7384-pr.5236.160e879 build
2a98fb1 billion-context@0.0.7288-pr.5236.1e77de8 build
a6a3e8f billion-context@0.0.7231-pr.5236.f442590 build
9c83b0a billion-context@0.0.7188-pr.5236.02ed0f7 build
c38777f billion-context@0.0.7090-pr.5236.95ab9a9 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5236.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 [PR] feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 Oct 30, 2026
ci-sync added 2 commits October 30, 2026 21:48
The timestamps feature bumped kernel/package.json 0.0.170 -> 0.0.171 but left
kernel/package-lock.json at 0.0.170, so the lock-sync job failed on the branch.
Lockfile now matches; only the two version fields change.
@ranxianglei ranxianglei changed the title [PR] feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 [PR] feat(#4313): Attach immutable local timestamps to every message and compression block — the model can now distinguish wh | feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 Oct 30, 2026
ci-sync and others added 4 commits October 30, 2026 22:44
… under the global legacy test baseline its sidecar never opens and the index reconstruction gate can never pass
The stamp node and tag/summary time rendering now key off a ports flag
(createCore({timestamps})); with the default core nothing is minted or
rendered, so default-off bytes stay identical to pre-feature master.
Prompt defaults revert to the exact pre-feature texts; the timestamped
wordings survive as exported variants hosts select behind their own
flag. Version bump 0.0.171 -> 0.0.172 (kernel-change: host-side opt-in
toggle for PR #5236 requires per-instance pipeline gating).
…feature wire bytes

compress.messageTimestamps / BILI_MESSAGE_TIMESTAMPS gate the display
side end-to-end: core pipeline ports (per-instance), fingerprint-span
receipt lines at the single applyRanges site, and the timestamped prompt
variants selected at every host prompt exit via messageTimestampSections.
Off is the default and stays byte-identical to pre-feature master. Docs
(CONFIGURATION.md + zh, website seeds regenerated by the official script,
coverage ok), storage-render goldens regenerated off-default (shape equals
pre-feature master; only state digests drift), new host toggle suite, and
config-surface seeds for both this key and sessions.hardMaxBytes.
@ranxianglei ranxianglei changed the title [PR] feat(#4313): Attach immutable local timestamps to every message and compression block — the model can now distinguish wh | feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 Oct 30, 2026
@ranxianglei ranxianglei changed the title feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 [PR] feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 Oct 30, 2026
…try, lockfile regen

- schema head is v10 (#4313 adds the immutable message-ts column); the
  master merge brought back v9 pins in issue4666/issue4677/issue4796 test files
  (mysql-contract 'expected 9 actual 10' + latent test-shard failures) — re-pin to 10
- register BILI_MESSAGE_TIMESTAMPS in LIFECYCLE.md §1 (opt-in feature flag,
  default off) — lifecycle-ledger-guard
- regenerate root package-lock.json with the exact CI command; the 0.0.172 bump
  commit missed the workspace version entry (0.0.171 → lock-sync red)

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review #5236(feat #4313:消息时间戳)

结论:功能代码审查通过;合并前必须先把分支同步到当前 master(见下,是真实的语义冲突,不是机械版本号冲突)。正式验证留给 CI。

合入前必须处理:分支落后当前 master(88 文件 / 19 个 commit)

master 已合入 2026-10-30 owner 版 #5188/#5140 sidecar 恢复(MaybeOpen 对瞬时环境故障会抛错、hadIntegrityQuickCheckFailure 熔断)与 #4665 archive/restore 携带 identity/retired_at。本分支携带的是同一 #5188 恢复工作的更早并行版(tryRecoverSidecar / isTransientContentionError / "MaybeOpen never throws" 假设)。本地尝试 merge origin/master 在下列文件出现真实内容冲突,已 abort,未推送任何改动:

  • src/persist.ts(同一段 tryRecoverSidecar,两侧注释与错误处理模型不同)
  • src/storage/store.ts / src/storage/mysql.ts / src/storage/shadow.ts
  • kernel/package.json、kernel/package-lock.json、根 package-lock.json

同步建议:rebase(或 merge)当前 master 后,丢弃本分支自带的旧版 #5188 代码(以 master 为准),只保留 #4313 的改动;最终 kernel 版本按 master 基线(现为 0.0.171)再 +1,并再生成 lockfile。

小问题

  1. PR 描述写 kernel/package.json 0.0.170→0.0.171,实际提交 bump 到 0.0.172(commit 9c83b0a「sync package-lock for the 0.0.172 bump」)。同步后请统一描述与实际版本。
  2. (随上解决)master 侧 kernel 已到 0.0.171,本分支从 0.0.170 直跳 0.0.172,lock 三方冲突即源于此。

功能审查(通过项)

  • 铸戳一次、永不回填:stamp-timestamps 节点只对真正新到的 id 铸 Date.now();已在 tsStamps 的 id 永不改写(持久化值 > carried 值);已知 id(byRaw 中已存在但无戳)保持无戳——不为老消息编造"今天"(kernel/src/compress.ts createStampTimestampsNode)。渲染路径只读存储值、从不读墙钟——前缀缓存不变量成立,且 verif 例外登记表给了同类于 createdAt 的理由(verif/exceptions.ts)。
  • 重启不重铸:cloneState / syncBlocks(kernel/src/sync.ts)/ 水合(src/state-hydration.ts mergeState)三处都携带 tsStamps;丢任何一处都会让重启后首轮全量重铸或静默掉属性,这里都防住了。
  • 存储层字节纪律:serializeForStore 先剥掉 ts 再算 identity/content_hash(src/storage/ingest.ts)——加时间戳不改内容身份;v10 可空列(sqlite+mysql 均 probe-first ALTER),存量行不回填(INSERT OR IGNORE 从不 UPDATE),新行缺逐条戳时取批内单一样例 batchTs;fork/archive 克隆行携带 ts(store.ts / mysql.ts 的 INSERT…SELECT 都含 ts 列)。
  • 回执跨度不谎报:blockCoverageTimes(src/stream.ts)要求覆盖区间全部 id(含边界 ref 解析)都有戳,任一缺失整体回退旧格式而不是收窄范围。
  • 默认关 = 字节兼容:提示词 DEFAULT/TIMESTAMPED 双变体、createCore 每实例一次性解析(#394)、render 节点独立门控、receipt span 同旗控制;存储层无条件记录所以后开可见完整历史——与描述一致。
  • 测试覆盖充分:铸一次不复铸 / 渲染确定性 / 老行原字节 / 跨块继承(kernel/tests/issue4313-message-timestamps.test.ts)+ toggle/fingerprint/migration v10/shadow/golden(golden drive 的墙钟掩码与既有 port/sid/tz 同类)。

本地定向验证(正式留给 CI,惯例)

kernel 定向套件(issue4313-message-timestamps + packs + pipeline)51/51 绿;宿主侧定向批(issue4313-toggle / issue4313-fingerprint / storage-unified / snapshot-shadow)33/33 绿。

是否可合并:暂缓——分支同步到 master(#5188 以 master 版为准)+ CI 绿之后再合;功能实现本身质量过关,未见逻辑缺陷。

dog added 2 commits October 31, 2026 08:55
… into pr5236w

# Conflicts:
#	kernel/package-lock.json
#	kernel/package.json
#	package-lock.json
#	src/persist.ts
#	src/storage/mysql.ts
#	src/storage/shadow.ts
#	src/storage/store.ts
#	tests/golden/storage-render/anthropic.json
#	tests/golden/storage-render/chat.json
#	tests/golden/storage-render/google.json
#	tests/golden/storage-render/responses.json
#	tests/issue5140-contention-not-disabling.test.ts
#	tests/issue5188-sidecar-recoverable-open.test.ts
#	tests/plugin-fork.test.ts
@ranxianglei

Copy link
Copy Markdown
Owner Author

跟进:分支已同步 master,冲突已解决,重新审查通过(2026-10-31)

做了什么

合并后验证(本地,正式 CI 在跑)

  • root build(kernel + pi-subagents + tsup + dist 检查)绿;tsc --noEmit 绿。
  • kernel 全套 1502/1502 绿。
  • 宿主侧定向批(golden-storage-render、storage-unified、snapshot-shadow、plugin-fork、issue5140、issue5188、issue4313-toggle/fingerprint、durable-tool-identity)109 pass / 0 fail(1 skip)。
  • 漏洞复核:无悬挂引用(旧 isTransientContentionError 引用清零);store.ts 主 ingest 路径不写 retired_at 是 master 既有设计(新行恒 active,retired 状态只由 historian UPDATE 产生);latch 计数器引用与 master 完全一致(保留用于 stats/面板)。

结论更新:上一条评论中的同步前置条件已满足,等本分支 CI 全绿即可合并。

@ranxianglei ranxianglei added the enhancement New feature or request label Oct 31, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

这个很重要但是是新功能,现在暂缓合并,需要再审查一次 从实证角度说明 关闭开关的时候 渲染是不变的

ranxianglei added a commit that referenced this pull request Oct 31, 2026
…tatusLine (kernel-guard Rule 2)

kernel/src changed five files; the PR body's "0.0.170 → 0.0.171" line was stale —
master is already at 0.0.171 (#5182) and this branch carried no bump, which trips
ci.yml kernel-guard Rule 2 (src changed without version bump → exit 1). Bump to
0.0.172 and re-sync both lockfiles. If #5236 lands first with 0.0.172, re-sync to
0.0.173 on merge.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复——但发现 #4313 存在两个并行实现 PR,需要你裁决二选一:

#5183(draft,10-30 00:19Z 先开) #5236(本 PR,10-30 12:13Z 后开)
开关 compress.timestamps 三级 compress 字段 compress.messageTimestamps + env BILI_MESSAGE_TIMESTAMPS
时间格式 UTC 分钟精度(与 issue 讨论结论一致) 本地时间秒精度(与讨论结论「UTC 分钟」相悖)
存储机制 kernel state 冻结(同 tokenSnapshot 纪律,不动 schema) 统一存储 v10,messages 加可空 ts 列(sqlite+mysql 迁移)
摘要时间 引擎追加 footer [folded 起→止 UTC] 范围 提示词让模型自写 "Occurred: 起–止" 行 + 指纹行带 [起–止]
体量 29 文件 +473/-34 55 文件 +1091/-120

两者设计互相冲突(格式/配置键/存储位置/摘要载体都不同),只能合一个。按先来后到 #5183 在先且仍是 draft、设计更贴近 #4313 讨论收敛的结论;#5236 已非 draft 且声明了 owner 拍板。我不关闭任何一个(关闭权在你)。

标签:pr、scope/kernel(主改面是 kernel 渲染与压缩提示词)、p3(opt-in 特性,默认关零影响)。

正在对 #5236 做完整代码审查(kernel + host + 存储迁移 + 测试有效性 + 回归清点),结论会带两 PR 对比与合并建议一并给出。

@ranxianglei ranxianglei changed the title [PR] feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生 feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生(默认关闭) Oct 31, 2026
@ranxianglei ranxianglei changed the title feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生(默认关闭) [PR] feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生(默认关闭) Oct 31, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

2026-10-31 复审 + 再同步说明(commit 3a05a42 merge / 1fa1f11 bump):

  1. CI 红已分诊:上一轮 head(2a98fb1a)四个红 shard 的失败点去重后只有两条——(master 上同样挂,见 [PR] Compression always failing and requiring many retries? Changed collapsed/protected region intervals to successful no-op | 压缩总是失败要重试很多次?已折叠/保护区区间改为成功 no-op 收据,并刷新 armed-but-silent 窗口的 ref 指针 (#4521) #5253 分诊时的两侧对照)和 (本地两侧都过,CI 环境 flake)。feat: Support precise timestamps for each message, allowing the model to recognize which messages are outdated and which | feat: 给每个消息支持精确时间,以让模型认识到什么消息是过期的什么是当前的新的 #4313 自身用例零失败,非本 PR 代码问题。
  2. master 又前进了:自上次同步点后新落入了 [PR] fix(plugin): "tool_calls": null deltas poisoned sawToolUse — degenerate empty stops never retried #5286/[PR] Compression always failing and requiring many retries? Changed collapsed/protected region intervals to successful no-op | 压缩总是失败要重试很多次?已折叠/保护区区间改为成功 no-op 收据,并刷新 armed-but-silent 窗口的 ref 指针 (#4521) #5253/[PR] fold idle session content stores between requests (#4788 phase 2) #5292/[PR] fix(storage): a live second writer no longer blocks startup — sharing a local-disk index.db is a supported topology (rev | fix(storage): 活着的第二写者不再阻止启动——共享一个本地盘 index.db 是受支持拓扑(撤回 #5213 的拒绝策略) #5294/[PR] fix(#5095): add per-run discriminator to the claude-anthropic derived-persona lane key #5163 五个合并。已再次 ——零冲突(persist.ts/stream.ts/preflight.ts 等重叠面全部自动合并干净)。
  3. 版本号按约定顺延:master 侧已由 [PR] Compression always failing and requiring many retries? Changed collapsed/protected region intervals to successful no-op | 压缩总是失败要重试很多次?已折叠/保护区区间改为成功 no-op 收据,并刷新 armed-but-silent 窗口的 ref 指针 (#4521) #5253 占用 0.0.172,本分支 kernel bump 改到 0.0.173,双 lockfile 重新生成,PR body 版本行同步更正。
  4. 同步后全量验证:kernel 全套 1507/1507;host 侧 24 个存储+compress+时间戳相关套件 163 pass / 0 fail / 4 skip;根 tsc 干净。

结论不变:功能代码通过、默认关闭零渲染字节变化、CI 绿即可人工合并(无 kernel boundary 问题,bump 已合规)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(上一条时间戳评论因 shell 反引号转义事故文本损坏,作废,以本条为准。)
2026-10-31 复审 + 再同步说明(commit 3a05a42 merge / 1fa1f11 bump):

  1. CI 红已分诊:上一轮 head(2a98fb1a)四个红 shard 的失败点去重后只有两条——「[PR] release v0.1.209 #4982 e2e draining same-lane predecessor」(master 上同样挂,见 [PR] Compression always failing and requiring many retries? Changed collapsed/protected region intervals to successful no-op | 压缩总是失败要重试很多次?已折叠/保护区区间改为成功 no-op 收据,并刷新 armed-but-silent 窗口的 ref 指针 (#4521) #5253 分诊时的两侧对照)和「claude-native hook e2e static URL」(本地两侧都过,CI 环境 flake)。feat: Support precise timestamps for each message, allowing the model to recognize which messages are outdated and which | feat: 给每个消息支持精确时间,以让模型认识到什么消息是过期的什么是当前的新的 #4313 自身用例零失败,非本 PR 代码问题。
  2. master 又前进了:自上次同步点后新落入了 [PR] fix(plugin): "tool_calls": null deltas poisoned sawToolUse — degenerate empty stops never retried #5286/[PR] Compression always failing and requiring many retries? Changed collapsed/protected region intervals to successful no-op | 压缩总是失败要重试很多次?已折叠/保护区区间改为成功 no-op 收据,并刷新 armed-but-silent 窗口的 ref 指针 (#4521) #5253/[PR] fold idle session content stores between requests (#4788 phase 2) #5292/[PR] fix(storage): a live second writer no longer blocks startup — sharing a local-disk index.db is a supported topology (rev | fix(storage): 活着的第二写者不再阻止启动——共享一个本地盘 index.db 是受支持拓扑(撤回 #5213 的拒绝策略) #5294/[PR] fix(#5095): add per-run discriminator to the claude-anthropic derived-persona lane key #5163 五个合并。已再次 merge origin/master——零冲突(persist.ts/stream.ts/preflight.ts 等重叠面全部自动合并干净)。
  3. 版本号按约定顺延:master 侧已由 [PR] Compression always failing and requiring many retries? Changed collapsed/protected region intervals to successful no-op | 压缩总是失败要重试很多次?已折叠/保护区区间改为成功 no-op 收据,并刷新 armed-but-silent 窗口的 ref 指针 (#4521) #5253 占用 0.0.172,本分支 kernel bump 改到 0.0.173,双 lockfile 重新生成,PR body 版本行同步更正。
  4. 同步后全量验证:kernel 全套 1507/1507;host 侧 24 个存储+compress+时间戳相关套件 163 pass / 0 fail / 4 skip;根 tsc 干净。

结论不变:功能代码通过、默认关闭零渲染字节变化、CI 绿即可人工合并(bump 已合规)。

@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.940-master.e193861 (rolling master tag).

npm install -g billion-context@0.0.940-master.e193861  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

1 participant