Repository navigation
Description
What would you like?
Harden createFileSystemSerdes() so checkpoint file references remain immutable, integrity-checked, and bound to the durable payload that produced them.
The current implementation writes JSON to a deterministic <entityId>.json path with writeFile, then stores an unversioned {"file": ...} envelope. A later serialization for the same entity can overwrite content referenced by an older checkpoint, and deserialization trusts the file path carried in the envelope.
Previous fixes addressed entity-ID filename encoding and overflow sizing. The remaining desired guarantees are:
- immutable file publication;
- a versioned, self-identifying envelope;
- durable-execution/entity ownership metadata;
- content-integrity and path-safety validation;
- continued support for
ALWAYS,OVERFLOW, previews, URI, and hash modes.
Possible Implementation
- Serialize once, compute a SHA-256 digest, and create a unique payload file with
writeFile(..., { flag: "wx" })or equivalent exclusive-create behavior. - Include an envelope marker/version, owner durable execution ARN, owner entity ID, payload type, file path, and content digest.
- On deserialization, validate the recognized envelope, expected execution directory and filename, base-path containment, symbolic links, owner rules, and content digest.
- Define explicit rules for safe incoming cross-execution references such as invoke input/results. Forwarded exceptions should be decoded and reserialized under the receiving child context when ownership changes.
- Preserve compatibility by continuing to read the current legacy envelope format for an appropriate migration period.
- Document storage lifecycle guidance because immutable publication can leave files that are no longer referenced by the latest checkpoint.
- Extend the cloud coverage tracked by Add cloud-based integration tests for filesystem serdes #710 with immutable publication and ownership/integrity scenarios.
Is this a breaking change?
No. Existing configuration should remain compatible, and legacy envelopes can continue to be readable.
Does this require an RFC?
Yes.
Additional Context
The Java SDK filesystem SerDes work in aws/aws-durable-execution-sdk-java#648 uses immutable publication, versioned ownership metadata, content hashes, containment checks, and symbolic-link rejection. This issue requests equivalent guarantees adapted to the async TypeScript implementation.
Activity
ParidelPooya commentedon Nov 1, 2026
Findings from a review of the current
createFileSystemSerdes()behavior:Overwriting the same file is mostly fine. Each serialization for an entity writes
<entityId>.json. Replay only reads the latest checkpoint, and the latest write holds the newest value, so last-write-wins matches what replay needs. For steps it's also safe: a step that dies before its checkpoint runs again and overwrites the file with a fresh result.There is one narrow gap, in
waitForCondition. Every check serializes its state under the samestepId, so every check writes the same file:For polling-style state (for example "latest job status") this is harmless, because the check just runs again. It only matters when the check accumulates (for example
count + 1), where one check can be applied twice. We haven't reproduced it yet; this is from reading the code.Suggested scope. Writing a unique file per serialization (for example with an exclusive create, as proposed) would close this gap. The other proposed guarantees (content digests, ownership metadata, symlink and containment checks) protect checkpoint data that is already trusted, so we'd treat them as separate, lower-priority hardening.
Keeping this open to track the unique-file fix.