Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/getreadable-metadata-only-key-lookup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': patch
---

Resolve a run's encryption key from metadata only, so `Run#getReadable()` and resuming a hook stored without a resume context no longer make the server resolve the run's whole input and output first.
6 changes: 6 additions & 0 deletions .changeset/inspect-metadata-only-key-lookup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@workflow/cli': patch
'@workflow/web': patch
---

Resolve a run's encryption key from metadata only when decrypting in `workflow inspect --decrypt` and the web UI, instead of making the server resolve the run's whole input and output for each lookup.
2 changes: 1 addition & 1 deletion docs/content/docs/v5/how-it-works/encryption.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ getEncryptionKeyForRun?(
): Promise<Uint8Array | undefined>;
```

Use `getEncryptionKeyForRun(run)` when the run entity already exists. Use `getEncryptionKeyForRun(runId, context?)` in runtime paths like `start()` where the run has not been created yet but the world may still need context such as `deploymentId`.
Use `getEncryptionKeyForRun(run)` when the runtime already holds the run entity. Use `getEncryptionKeyForRun(runId, context?)` everywhere else: in paths like `start()` where the run does not exist yet, and in paths that have a run id but no reason to load the run's payloads, such as resolving the key for a stream read. Either way `context` carries whatever the World needs to find the key, such as `deploymentId`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Nit

The matching JSDoc on the interface, packages/world/src/interfaces.ts:789-794, still says the (runId, context?) overload is "Used when the run entity is not locally available, such as start() before run creation or a forwarded writable stream…". That is the narrower wording this sentence replaces, and World authors read the JSDoc too. It would be good to update it the same way: "everywhere else, including paths that have a run id but no reason to load its payloads (e.g. stream reads)".

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Nit

Addressed in c972931. The World.getEncryptionKeyForRun JSDoc in packages/world/src/interfaces.ts now matches the docs: the run-object form is for when the runtime already holds the run, and (runId, context?) is for everywhere else (stream reads, forwarded writables, hook resumption). It also says outright that both forms must resolve the same key. The change is JSDoc only, so there is no changeset for @workflow/world.


To add encryption support to a custom `World`:

Expand Down
87 changes: 87 additions & 0 deletions packages/cli/src/lib/inspect/output.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import {
listStreamsByRunId,
showEvent,
showStep,
showStream,
} from './output.js';

const makeRun = (overrides: Partial<WorkflowRun> = {}): WorkflowRun =>
Expand Down Expand Up @@ -1605,3 +1606,89 @@ describe('showStep', () => {
);
});
});

describe('--decrypt key lookup', () => {
// The key lookup needs only the run's deploymentId, which survives
// `resolveData: 'none'`; the default ('all') would make the World resolve
// the run's whole input and output just to supply that field (#4645).
const makeWorld = (
getEncryptionKeyForRun: World['getEncryptionKeyForRun'] = vi
.fn()
.mockResolvedValue(undefined)
) =>
({
runs: { get: vi.fn().mockResolvedValue(makeRun()) },
steps: {
get: vi.fn().mockResolvedValue({
runId: 'run-1',
stepId: 'step-1',
stepName: 'step',
status: 'completed',
input: undefined,
output: undefined,
attempt: 1,
createdAt: new Date('2025-01-01'),
updatedAt: new Date('2025-01-01'),
}),
},
streams: {
get: vi
.fn()
.mockResolvedValue(new ReadableStream({ start: (c) => c.close() })),
},
getEncryptionKeyForRun,
}) as unknown as World;

it('reads the run metadata only when resolving a resource key', async () => {
const world = makeWorld();
vi.spyOn(process.stdout, 'write').mockImplementation(() => true);

await showStep(world, 'step-1', {
runId: 'run-1',
decrypt: true,
json: true,
});

expect(world.runs.get).toHaveBeenCalledWith('run-1', {
resolveData: 'none',
});
expect(world.getEncryptionKeyForRun).toHaveBeenCalledWith('run-1', {
deploymentId: 'dep-1',
});
});

it('reads the run metadata only when resolving a stream key', async () => {
const world = makeWorld();
vi.spyOn(process.stdout, 'write').mockImplementation(() => true);
vi.spyOn(logger, 'info').mockImplementation(() => {});

await showStream(world, 'default', {
runId: 'run-1',
decrypt: true,
json: true,
});

expect(world.runs.get).toHaveBeenCalledWith('run-1', {
resolveData: 'none',
});
expect(world.getEncryptionKeyForRun).toHaveBeenCalledWith('run-1', {
deploymentId: 'dep-1',
});
});

it('reads no run for a stream when the World cannot encrypt', async () => {
const world = makeWorld();
delete (world as { getEncryptionKeyForRun?: unknown })
.getEncryptionKeyForRun;
vi.spyOn(process.stdout, 'write').mockImplementation(() => true);
vi.spyOn(logger, 'info').mockImplementation(() => {});

await showStream(world, 'default', {
runId: 'run-1',
decrypt: true,
json: true,
});

expect(world.runs.get).not.toHaveBeenCalled();
});
});
25 changes: 17 additions & 8 deletions packages/cli/src/lib/inspect/output.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@ import { resolveWorkflowNameFilter } from './workflow-name.js';
* Create an EncryptionKeyResolver from a World instance.
* Returns null if decrypt is false; encrypted data will show as a placeholder.
*
* The resolver fetches the full WorkflowRun (cached per runId) so that the
* World can inspect deployment-specific fields for key resolution.
* The resolver reads the run's metadata (cached per runId) for the
* deploymentId the World needs to find the key. It reads metadata only: the
* default `resolveData` would also resolve the run's whole input and output.
*/
function createResolver(world: World, decrypt: boolean): EncryptionKeyResolver {
if (!decrypt) return null;
Expand All @@ -44,8 +45,10 @@ function createResolver(world: World, decrypt: boolean): EncryptionKeyResolver {
let cached = cache.get(runId);
if (!cached) {
cached = world.runs
.get(runId)
.then((run) => world.getEncryptionKeyForRun!(run));
.get(runId, { resolveData: 'none' })
.then(({ deploymentId }) =>
world.getEncryptionKeyForRun!(runId, { deploymentId })
);
cache.set(runId, cached);
}
return cached;
Expand Down Expand Up @@ -1119,13 +1122,19 @@ export const showStream = async (
const rawStream = await world.streams.get(opts.runId, streamId);

// Only resolve the encryption key when --decrypt is passed and --run is provided.
// We fetch the full WorkflowRun object so that getEncryptionKeyForRun has
// access to the deploymentId (needed for API-based key resolution).
// The run is read for its deploymentId (needed for API-based key
// resolution), metadata only: its payloads play no part in the lookup.
let encryptionKey: EncryptionKeyParam;
if (opts.decrypt && opts.runId) {
encryptionKey = (async () => {
const run = await world.runs.get(opts.runId!);
const rawKey = await world.getEncryptionKeyForRun?.(run);
if (!world.getEncryptionKeyForRun) return undefined;
const runId = opts.runId!;
const { deploymentId } = await world.runs.get(runId, {
resolveData: 'none',
});
const rawKey = await world.getEncryptionKeyForRun(runId, {
deploymentId,
});
// Full capability, so sealed ('encp') stream frames written by other
// runs are readable too, not just the run's own symmetric frames.
return rawKey ? await deriveRunPayloadKeys(rawKey) : undefined;
Expand Down
10 changes: 6 additions & 4 deletions packages/core/src/runtime/get-hook-by-token.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -165,10 +165,12 @@ describe('getHookByToken (lazy metadata)', () => {

await found.metadata;

expect(runsGet).toHaveBeenCalledWith(hook.runId);
// The fallback reuses the run it just fetched rather than re-resolving by
// runId + deploymentId.
expect(getEncryptionKeyForRun).toHaveBeenCalledWith(run);
// Metadata only: hydrating needs the run's deploymentId, not its
// payloads (#4645).
expect(runsGet).toHaveBeenCalledWith(hook.runId, { resolveData: 'none' });
expect(getEncryptionKeyForRun).toHaveBeenCalledWith(hook.runId, {
deploymentId: run.deploymentId,
});
});

it('surfaces a hydration failure on access, not on lookup', async () => {
Expand Down
9 changes: 7 additions & 2 deletions packages/core/src/runtime/resume-hook.fast-path.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -128,11 +128,16 @@ describe('resumeHook (resumeContext fast path)', () => {
specVersion: SPEC_VERSION_CURRENT,
} as unknown as WorkflowRun;
const runsGet = vi.fn().mockResolvedValue(run);
const { queue } = makeWorld(hook, { runsGet });
const { queue, getEncryptionKeyForRun } = makeWorld(hook, { runsGet });

await resumeHook(hook.token, { foo: 'bar' });

expect(runsGet).toHaveBeenCalledWith(hook.runId);
// Metadata only: the fallback needs the run's deploymentId and status,
// not its payloads (#4645).
expect(runsGet).toHaveBeenCalledWith(hook.runId, { resolveData: 'none' });
expect(getEncryptionKeyForRun).toHaveBeenCalledWith(hook.runId, {
deploymentId: 'deployment_fallback',
});
expect(queue).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({ runId: hook.runId }),
Expand Down
6 changes: 5 additions & 1 deletion packages/core/src/runtime/resume-hook.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,11 @@ describe('resumeHook', () => {

await resumeHook('order:1', { approved: true });

expect(getEncryptionKeyForRun).toHaveBeenCalledWith(run);
// The fallback read only the run's metadata, so the key is resolved by
// runId + the deploymentId it carried (#4645).
expect(getEncryptionKeyForRun).toHaveBeenCalledWith(run.runId, {
deploymentId: run.deploymentId,
});
expect(peekFormatPrefix(capturedPayload(createEvent))).toBe(
SerializationFormat.ENCRYPTED
);
Expand Down
43 changes: 24 additions & 19 deletions packages/core/src/runtime/resume-hook.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
SPEC_VERSION_LEGACY,
SPEC_VERSION_SUPPORTS_COMPRESSION,
type WorkflowInvokePayload,
type WorkflowRun,
type WorkflowRunWithoutData,
type Hook as WorldHook,
} from '@workflow/world';
import { monotonicFactory } from 'ulid';
Expand Down Expand Up @@ -129,11 +129,14 @@ interface HookLookup {
interface HookResumeInfo {
resumeContext: HookResumeContext;
source: 'hook' | 'run_fallback';
run?: WorkflowRun;
run?: WorkflowRunWithoutData;
}

/** Derive a resume context from a full run (fallback for pre-`resumeContext` hooks). */
function resumeContextFromRun(run: WorkflowRun): HookResumeContext {
/**
* Derive a resume context from a run's metadata (fallback for
* pre-`resumeContext` hooks). Nothing here reads the run's payloads.
*/
function resumeContextFromRun(run: WorkflowRunWithoutData): HookResumeContext {
const coreVersion = run.executionContext?.workflowCoreVersion;
const traceCarrier = run.executionContext?.traceCarrier;
const hookResumeInputVersion = run.executionContext?.hookResumeInputVersion;
Expand Down Expand Up @@ -172,7 +175,12 @@ async function resolveHookResumeInfo(
if (hook.resumeContext) {
return { resumeContext: hook.resumeContext, source: 'hook' };
}
const run = await (await getWorldLazy()).runs.get(hook.runId);
// Metadata only: the resume context, the terminal-status check, and the key
// lookup need none of the run's input or output, and the default
// `resolveData` ('all') would make the World resolve both (#4645).
const run = await (await getWorldLazy()).runs.get(hook.runId, {
resolveData: 'none',
});
return {
resumeContext: resumeContextFromRun(run),
source: 'run_fallback',
Expand All @@ -184,19 +192,18 @@ async function resolveHookResumeInfo(
* Resolve the run's symmetric key for a payload WRITE, as a bare `CryptoKey`
* (`importKey`): the `encr` write fallback used when the run published no
* public key to seal to. Writing needs only the AES key, not the read-side
* keypair. On the fast path this needs only `runId` + `deploymentId` (no run
* entity); on the fallback path the already fetched run is reused.
* keypair. Both paths resolve it by `runId` + the resume context's
* `deploymentId`; on the fallback path that context came from the run's
* metadata, so no payload is ever read for the key.
*/
async function resolveHookEncryptionKey(
hook: ResumableHook,
info: HookResumeInfo
): Promise<Awaited<ReturnType<typeof importKey>> | undefined> {
const world = await getWorldLazy();
const rawKey = info.run
? await world.getEncryptionKeyForRun?.(info.run)
: await world.getEncryptionKeyForRun?.(hook.runId, {
deploymentId: info.resumeContext.deploymentId,
});
const rawKey = await world.getEncryptionKeyForRun?.(hook.runId, {
deploymentId: info.resumeContext.deploymentId,
});
return rawKey ? await importKey(rawKey) : undefined;
}

Expand Down Expand Up @@ -230,13 +237,11 @@ function withLazyMetadata(raw: WorldHook): HookLookup {
const hydrate = async (): Promise<unknown> => {
const world = await getWorldLazy();
const info = await resolveHookResumeInfo(raw);
// On the fast path this resolves the key by runId + deploymentId (no run
// read); on the fallback path it reuses the already-fetched run.
const rawKey = info.run
? await world.getEncryptionKeyForRun?.(info.run)
: await world.getEncryptionKeyForRun?.(raw.runId, {
deploymentId: info.resumeContext.deploymentId,
});
// Resolved by runId + deploymentId on both paths: the fast path reads no
// run, and the fallback path read only the run's metadata.
const rawKey = await world.getEncryptionKeyForRun?.(raw.runId, {
deploymentId: info.resumeContext.deploymentId,
});
encryptionKey = rawKey ? await deriveRunPayloadKeys(rawKey) : undefined;
return await hydrateStepArguments(
serialized as any,
Expand Down
Loading
Loading