Repository navigation
RFC 994: Structured design context in synthesized templates - #981
Conversation
iliapolo
left a comment
There was a problem hiding this comment.
@satyakigh Are you also looking for feedback on the schema itself, or just the CDK part of things?
35cad9a to
12543b4
Compare
iliapolo
left a comment
There was a problem hiding this comment.
Also please provide the full public API of all classes we are introducing (in TypeScript form). Right now its kind of scattered between examples and hard to get a full picture.
We can then approve that API and merge it as stable from the get go.
| contributing evidence in `cite` and `note`. | ||
|
|
||
| For example, a tool that lifts `why` from a comment writes `src: COMMENT` and | ||
| `cite: 'lib/queue.ts:42'`; when the author reviews and accepts it, `src` becomes `AUTHORED` |
There was a problem hiding this comment.
You mean "src should become AUTHORED"? As in, the PR author should make a change after getting approval?
There was a problem hiding this comment.
Yes. It now reads: "the author (or the tool, on the author's confirmation) should change src to AUTHORED and keep cite"
There was a problem hiding this comment.
Thinking about this again - isn't every change to the template eventually human reviewed? and is therefore AUTHORED? when will my deployed template change without human review?
|
@satyakigh Im happy with the API - this can now move onto final comments period. Can you please create a tracking issue similar to #949 where we can apply the correct labels? See https://github.com/aws/aws-cdk-rfcs#1-tracking-issue for more details. Thanks! |
Co-authored-by: Eli Polonsky <Eli.polonsky@gmail.com>
|
Updated RFC to the correct number, created #994 |
rix0rrr
left a comment
There was a problem hiding this comment.
After reading this, I do think we would benefit from a machine executable mechanism to relate various properties to each other. Could be an expression language, could be some CDK functions that a user strings together.
What I don't think we need is a more elaborate and noisy way to write agent instructions. Source code comments or Markdown files will do just fine there.
EDIT: After going through the doc and getting worked up about how this is not adding any value to CDK users, I now realize this is a CloudFormation standard that we are trying to get people to adopt. So I think the motivation for this feature can be summarized as "comments are important, and JSON cannot have code comments in it". I buy that and I agree with it, but it only applies to templates being authored by hand. Which does not apply to CDK users though.
So the reason to implement this is to have feature parity with CFN?
| command generates a CloudFormation template. This RFC adds two application programming | ||
| interfaces (APIs) to `aws-cdk-lib`, `ResourceMetadataContext` and `TemplateMetadataContext`, | ||
| that add structured design information under the dedicated | ||
| `com.aws.cloudformation.Context` metadata key. People and automated tools, including |
There was a problem hiding this comment.
Very different from other keys that exist. We stepped out of CloudFormation world into Java world.
A bit jarring tbh.
| ResourceMetadataContext.of(queue).add({ | ||
| why: 'buffer order events asynchronously; 14-day retention meets compliance requirements', | ||
| must: ['VisibilityTimeout must be at least six times the Lambda timeout to avoid duplicate processing'], | ||
| mutable: ContextMutability.CHANGE_WITH_CONSTRAINTS, |
There was a problem hiding this comment.
What is this field about? What can change?
There was a problem hiding this comment.
mutable is the team's default tolerance for changing the resource, and mutability overrides it per property. Both describe what the team considers safe to change
| and how safely each resource can change. It often lives only in source comments, | ||
| the hierarchy of CDK constructs, or the author's knowledge, and is lost when the `cdk synth` |
There was a problem hiding this comment.
Wait -- source is the exact right place to put it.
If knowledge lives "just in the author's head", that is obviously bad, but it's the same as any other programming language. We are not proposing to inject source comments into Java .class files. Nor are we saying that any place of documentation is as good as any other place.
So you need to make it plausible, to me as a reader, why the template is the right location for this information. I cannot just take this as a given, and "hand-wave agents" is not enough. We already have a way to embed the source location for an application into the template, so an agent can trace its way back from a template to the source code that produced it, and find the source comments there. Give me examples of scenarios that cannot be solved in other, better ways.
I'm also somewhat concerned about template size. There are limits to the template. You also haven't explained why, if you MUST have this, why it shouldn't live in a separate file in S3 somewhere. (And then, if it's a separate file in S3, why that couldn't be a Markdown file that was automatically extracted from the source code and comments in there by your agent)
| must: ['VisibilityTimeout must be at least six times the Lambda timeout to avoid duplicate processing'], | ||
| mutable: ContextMutability.CHANGE_WITH_CONSTRAINTS, | ||
| mutability: { | ||
| QueueName: ContextMutability.MUST_NEVER_CHANGE, |
There was a problem hiding this comment.
This information can be obtained from the CloudFormation resource schema, no?
Also, this talks about QueueName (CloudFormation property name) rather than queueName (CDK property name) so knowingly pierces the abstraction layer.
There was a problem hiding this comment.
The resource schema says QueueName is create-only, so changing it replaces the queue; this specifies whether the team can live with a replacement.
| To reach more than the primary resource, set `propagate: true`. Propagation replaces | ||
| `defaultChild` selection entirely: the declaration applies to every resource beneath the | ||
| scope, helpers included, and only a `PropagationFilter` narrows it, by resource type: |
There was a problem hiding this comment.
In that case, don't do 2 fields. Classes/objects can fully represent the tagged union you're trying to model here.
// ☝️
{
propagate: true,
propagationFilter: PropagationFilter.includeResourceTypes(...),
}
// 👉
{
propagate: Propagate.toResourceTypes([...]),
}
// Also available
Propagate.toDefaultChild() // Default
Propagate.toSpecificChildren([...])
Propagate.toAllChildren({ except: [...] })
// More?| **Evaluation.** The evaluation tested whether added design information changes how a tool updates a | ||
| CloudFormation template. The latest evaluation used 33 tasks, ran each condition three | ||
| times, and scored expected outcomes with repeatable text checks: |
There was a problem hiding this comment.
So, the evaluation was:
- A CDK app generates a template
- The agent was then instructed to make modifications to the template directly
That is not a realistic experiment. No user would do that or instruct an agent to do that. If you are working with CDK, you are working with CDK, not editing the template out from under CDK.
| added information. Those tasks raise every score and make the differences between | ||
| conditions appear smaller. |
There was a problem hiding this comment.
This is phrased as if this is somehow "unfair" to the new metadata context.
But it is just the reality of working with CloudFormation/in AWS. It also applies to real world tasks. So if anything is representative, this is.
| * The scoring checks for expected words or phrases. It does not evaluate the complete | ||
| quality of an answer, which helps explain the relatively high score without added | ||
| information. |
There was a problem hiding this comment.
Read as "Our eval scoring is ¯\(ツ)/¯ so that's why the gap is smaller than it could have been. But trust us on the high numbers, those are definitely realistic and meaningful!"
| * Comments (93.56%) and structured metadata without special instructions (94.70%) performed | ||
| similarly. Most of the measured benefit came from providing design information in any | ||
| form. |
| * The evaluation shows that added information helps, but does not by itself prove that | ||
| structured fields outperform comments. Structured fields are still needed because AWS | ||
| CDK does not preserve source comments in generated templates, and named fields can be | ||
| retrieved individually by tools. |
There was a problem hiding this comment.
Read as "Source comments in CDK don't help in the unrealistic task of making changes without those source code comments".
Too bad user's daily jobs mostly consist of realistic tasks.
| ##### Conflict with manually added context | ||
|
|
||
| `com.aws.cloudformation.Context` is a normal metadata key, so callers can also write it | ||
| directly with `CfnResource.addMetadata()`. If manually added information and a | ||
| metadata-context API target the same resource and key, template generation fails instead of | ||
| silently overwriting the caller's information: |
There was a problem hiding this comment.
Why is this the better behavior? What if a third-party construct declares this metadata? Can I now not override it anymore?
| must: ['execution roles must keep the organization permissions boundary'], | ||
| }, { | ||
| propagate: true, | ||
| propagationFilter: PropagationFilter.excludeResourceTypes(['AWS::Lambda::Function']), |
There was a problem hiding this comment.
Mixins already offer a filtering system dubbed ConstructSelector. Any reason we cannot reuse and extend that system?
There was a problem hiding this comment.
Done, updated the design
| On that queue's `AWS::SQS::Queue`, the closer `why` wins and the `must` entries combine, | ||
| Stack entry first. Other queues in the Stack render only the Stack declaration. |
There was a problem hiding this comment.
I think this is right, but it directly contradicts statements elsewhere claiming that silently overriding is bad.
There was a problem hiding this comment.
Agreed, cleared up the wording
| ```ts | ||
| declare const app: App; | ||
|
|
||
| // Each Stage is a separate cloud assembly and declares its own context. |
There was a problem hiding this comment.
this statement is not correct
| ```ts | ||
| declare const legacyBucket: s3.Bucket; | ||
|
|
||
| // This legacy bucket is exempt from the customer managed AWS KMS key requirement. | ||
| ResourceMetadataContext.of(legacyBucket).add({ | ||
| why: 'legacy public assets; migration tracked separately; approved encryption exception', | ||
| }, { | ||
| inheritAncestorContext: false, | ||
| }); | ||
| ``` |
There was a problem hiding this comment.
Okay so you want to support local overrides for values set further up in the construct tree, while also doing magically merging. I am not convinced this syntax makes this clear and easy to use.
Maybe there's a difference between .add() and .set() (and therefore we also need .clear())
| }); | ||
| ``` | ||
|
|
||
| ##### Helper resources |
There was a problem hiding this comment.
This feels like a subsection of propgation, but is currently presented as its own thing.
| trust: { | ||
| src: ContextTrustSource.INFER, | ||
| conf: ContextTrustConfidence.LOW, | ||
| cite: 'service/handler.ts:87', |
There was a problem hiding this comment.
This screams of documentation rot. If we cannot solve this automatically for users, it even feels dangerous.
There was a problem hiding this comment.
The extra fields cite, note are meant more for tools that generate context, not required for humans authoring the context. This was a bad example of cite, but it can be anything to URLs, external links, etc.
| Add resource-level information to a construct scope with `ResourceMetadataContext`. A scope | ||
| is a node in the CDK construct hierarchy. By default, the information is written to the | ||
| scope's *primary resource*: the CloudFormation resource reached by following CDK's | ||
| [`defaultChild`](https://docs.aws.amazon.com/cdk/api/v2/docs/constructs.Node.html#defaultchild) | ||
| property. For example, the primary resource of an Amazon Simple Queue Service (Amazon SQS) | ||
| `sqs.Queue` construct is its `AWS::SQS::Queue` resource. Automatically created helper | ||
| resources, such as AWS Identity and Access Management (IAM) roles, policies, and | ||
| log-retention custom resources, are not selected by default. |
There was a problem hiding this comment.
This is not answering a fundamental question: Is the metadata added to the scope imperatively or declaratively? Imperative ~= Mixins. Declarative ~= Aspects.
The huge difference is when we add to an upper scope like a stack. Will the metadata be applied to only the constructs currently in the stack or all future constructs added later in code?
Which ever you choose, justify the choice and this will guide the further design. I also would expect you lean into the respective system and use it under the hood.
There was a problem hiding this comment.
It's declarative, add records the declaration on the scope and an Aspect resolves it at synthesis, so a declaration on a Stack covers resources added to the stack later in code.
| /** Inherit context merged from ancestor scopes. @default true */ | ||
| readonly inheritAncestorContext?: boolean; | ||
| export interface CfnResourceMetadataContextOptions { | ||
| /** Selects the constructs whose CloudFormation resources receive the declaration. @default ConstructSelector.onlyItself() */ |
There was a problem hiding this comment.
Default should be cfnResource to clearly communicate that by default we go into the default child - and if it doesn't exist thats an error.
| call `clear()` on it. The ancestor's selector decides which resources are candidates, and | ||
| `clear()` removes every candidate beneath the cleared scope, whatever the ancestor | ||
| selected; declarations on the cleared scope or beneath it still apply, and the result is the | ||
| same whether `clear()` runs before or after `add()`. The following example refers to a |
There was a problem hiding this comment.
I dont see how the call order doesn't make a difference - I think it should.
| CfnResourceMetadataContext.of(legacyBucket).clear(); | ||
| CfnResourceMetadataContext.of(legacyBucket).add({ |
There was a problem hiding this comment.
Lets add a set method, then clear can be implemented with an empty set.
|
|
||
| // Template generation throws ValidationError because two authoring methods target one key. | ||
| // A declaration on the queue, whose primary resource this is, overrides the directly | ||
| // written `why`; the `must` entries combine. |
There was a problem hiding this comment.
This isn't specific to direct addMetadata calls - what happens if there are 2 add calls on the same scope? because we can't combine why - we should throw an error and let the customer decide (i.e use set or dont call add).
| and `trust`; `must` and `deps` are unioned and de-duplicated; `mutability` merges per | ||
| property. `inheritAncestorContext: false` discards ancestor context for that scope. | ||
| * `TemplateMetadataContext.of(stack).add()` merges repeated calls: later `arch` and `owner` | ||
| property. Repeated `add()` calls on the same scope merge in call order, later wins for |
There was a problem hiding this comment.
Decided offline to drop add and the whole merging mechanism. Instead, set will always override (same as direct addMetadata with the same key`).
We will add merging capabilities with add later on if they are needed
|
Addressing Oct 20 feedback
|
discussed internally
…(MetadataContext) (#38381) ### Issue # (if applicable) RFC tracking issue: aws/aws-cdk-rfcs#994 Accepted RFC: aws/aws-cdk-rfcs#981 ### Reason for this change CloudFormation templates preserve deployed configuration but usually lose the rationale, invariants, change-safety guidance, and provenance available in CDK source. CloudFormation defines the advisory `com.aws.cloudformation.Context` metadata schema for preserving that information, but CDK users need a typed authoring API. ### Description of changes - Adds `CfnResourceMetadataContext` for setting or clearing Context on selected `CfnResource` constructs during synthesis. - Adds `CfnTemplateMetadataContext` for setting or clearing template-level Context. - Mirrors the published resource, template, mutability, reference, and trust fields without adding another vocabulary. - Uses `IConstructSelector` for explicit resource targeting. The default selects a `CfnResource` scope or its immediate `CfnResource` default child. - Uses complete-value replacement rather than implicit merging. Aspect priority and scope determine which overlapping operation applies last. - Preserves `NestedStack` behavior and does not cross `Stage` assembly boundaries. - Rejects invalid trust data, redundant sparse mutability entries, selectors that match no `CfnResource`, and unresolved values in top-level template Context. - Adds public API documentation, README examples, unit coverage, and an integration snapshot. The public API and stable-release plan were approved in RFC 994. The approved surface keeps resource Context separate from targeting options. jsii generates Python as `set(context, *, priority=None, selector=None)` and Java as `set(context)` or `set(context, options)`. RFC 994 also requires rejecting a `mutability` entry that repeats the resource-level `mutable` default so the map remains sparse. Existing applications synthesize unchanged unless they call the new APIs. ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
This RFC proposes structured design context in synthesized templates. See #994 for tracking and status.
The public APIs are signed off by @iliapolo.
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache-2.0 license