Skip to content

docs(kiro): narrow the validate-template prohibition to provider-owned constraints - #4932

Merged
kddejong merged 1 commit into
aws-cloudformation:mainfrom
kddejong:docs/validate-template-control-plane
Oct 26, 2026
Merged

kddejong merged 1 commit into
aws-cloudformation:mainfrom
kddejong:docs/validate-template-control-plane

Conversation

@kddejong

Copy link
Copy Markdown
Contributor

Problem

.kiro/ tells agents that aws cloudformation validate-template is not evidence for a constraint. For a property-level constraint that is right: validate-template invokes no resource provider, so it cannot establish a property value, limit, enum or cross-property rule.

The wording over-reached. It also covered template-level structure, which the CloudFormation control plane parses itself — exactly what validate-template does check. For that class, validate-template returns the same error CreateStack does, and it is free and leaves nothing behind.

This surfaced on #4854. The reporter cited validate-template for a parameter type, our own agent context said that was not evidence, and the reporter was right. These files are read on every agent session, so the next agent would have dismissed correct evidence the same way.

Evidence

Recorded us-east-1, 2026-10-26, on this template:

Parameters:
  P:
    Type: List<AWS::SSM::Parameter::Value<String>>
Resources:
  W:
    Type: AWS::CloudFormation::WaitConditionHandle
$ aws cloudformation validate-template --region us-east-1 --template-body file://t.yaml
An error occurred (ValidationError) when calling the ValidateTemplate operation:
Template format error: List of AWS::SSM::Parameter::Value is not supported.

$ aws cloudformation create-stack --region us-east-1 --stack-name vt-probe-1 \
    --template-body file://t.yaml --parameters ParameterKey=P,ParameterValue=x
An error occurred (ValidationError) when calling the CreateStack operation:
Template format error: List of AWS::SSM::Parameter::Value is not supported.

$ aws cloudformation describe-stacks --region us-east-1 --stack-name vt-probe-1
An error occurred (ValidationError) ... Stack with id vt-probe-1 does not exist

Identical error code and message from both APIs, and no stack created. A parameter type is parsed by the control plane, not by a resource provider, which is why the two agree here and do not agree on a property constraint.

What changed

Three files, one paragraph each, kept short because they are session context.

  • .kiro/context/verification.md — reframed from "is NOT evidence" to "is scoped to the control plane", with both halves stated and the measured parameter-type case recorded.
  • .kiro/context/false-positives.md (doctrine item 4) — same distinction, compressed; the measured case is not repeated here.
  • .kiro/steering/rule-development.md (Validation Approach step 1) — the one-clause version now says "for a property constraint" instead of forbidding the tool outright.

"Never cite it as a probe" is kept for the property case.

Not changed

Verification

pre-commit run --all-files     # all hooks pass; the three .md files are untouched by every fixer
python -m pytest test          # 19 pre-existing failures, identical with these edits stashed:
                               # 16 are missing optional extras (pydot, pygraphviz, junit_xml,
                               # sarif_om), the rest are pricing-maintenance assertions

The dependencies (ubuntu-latest, 3.10) job fails on a pip-audit CVE in filelock, pre-existing on main and unrelated to this change.

…d constraints

validate-template invokes no resource provider, so it cannot establish a
property value, limit, enum or cross-property rule. That reasoning does not
extend to template-level structure, which the control plane itself parses and
which validate-template is therefore testing directly.

Recorded us-east-1, 2026-10-26: a parameter of
Type: List<AWS::SSM::Parameter::Value<String>> draws the identical
"Template format error: List of AWS::SSM::Parameter::Value is not supported."
from validate-template and from create-stack, and no stack is created.

Raised by issue aws-cloudformation#4854, where the reporter cited validate-template for a
parameter type and our own context said that was not evidence. The
deployment-first evidence ranking from aws-cloudformation#4867 is unchanged.
@codecov

codecov Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.68%. Comparing base (ccefd71) to head (695bf8c).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4932      +/-   ##
==========================================
- Coverage   94.70%   94.68%   -0.03%     
==========================================
  Files         441      441              
  Lines       16058    16067       +9     
  Branches     3113     3116       +3     
==========================================
+ Hits        15208    15213       +5     
- Misses        468      470       +2     
- Partials      382      384       +2     
Flag Coverage Δ
unittests 94.67% <ø> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kddejong
kddejong marked this pull request as ready for review October 26, 2026 20:45
@kddejong
kddejong merged commit 0a02f1b into aws-cloudformation:main Oct 26, 2026
23 of 24 checks passed
@kddejong
kddejong deleted the docs/validate-template-control-plane branch October 26, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant