Repository navigation
fix: stop Runner.__init__ exiting on an unresolvable template glob - #4966
Merged
kddejong merged 1 commit intoOct 30, 2026
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4966 +/- ##
=======================================
Coverage 94.73% 94.73%
=======================================
Files 442 442
Lines 16094 16102 +8
Branches 3123 3125 +2
=======================================
+ Hits 15247 15255 +8
Misses 467 467
Partials 380 380
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
kddejong
marked this pull request as ready for review
October 30, 2026 22:35
Runner.__init__ reported an unresolvable --template glob by calling _cli_output, which ends in sys.exit. Constructing a Runner therefore terminated any process that embeds cfn-lint as a library, four lines above the schema patching that aws-cloudformation#4911 fixed for the same reason. cfnlint.api lint(), lint_file() and lint_by_config() all died the same way. Record the error during construction and report it from cli() through the unchanged _cli_output path, so the CLI keeps its exit code and byte-level output for every --format, and raise TemplatesNotFoundError from run() so a library caller gets a catchable CfnLintExitException instead of a dead process. The report has to come from a Runner that has loaded its rules: JUnitFormatter returns None for an empty rule set and SARIFFormatter raises KeyError, so reporting from main() would have emitted nothing for --format junit and crashed for --format sarif. Also export DuplicateRuleError, ConfigFileError and SchemaPatchError from cfnlint.runner, which already exported the other two subclasses and is where a library caller looks for them.
kddejong
force-pushed
the
fix/runner-init-library-safety
branch
from
October 30, 2026 22:36
3c15987 to
c80b186
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
Follows #4911, which stopped schema patching from calling
sys.exit(1)in a library caller's process. Two gaps remained.Description of changes
1.
Runner.__init__still terminated the process. Four lines above the code #4911 fixed, an unresolvable--templateglob was reported by calling_cli_output, which ends insys.exit:So
Runner(ConfigMixIn([...]))killed the host process, and with itcfnlint.api.lint(),lint_file()andlint_by_config().except Exceptioncannot catch this —SystemExitis aBaseException— so the only reliable evidence is whether a statement after the call runs. Onmainit does not:lint_by_configdoes reach this path:config.validate()reads the raw arguments and passes, thenRunnerglobs them. On this branch all three API functions raise instead:with the message text unchanged.
lint()no longer dies at all — it lints the string it was given, since the glob is irrelevant to it.Why the report moved to
cli()rather thanmain(). The error is a formatted match, not a bare log line, so--format jsoncurrently emits a valid JSON match object. Reporting it frommain()(the shape #4911 used) is not possible without regressing that, because the formatters need the loaded rule set:JUnitFormatter.print_matchesbeginsif not rules: return None→--format junitwould emit nothing at allSARIFFormatter.print_matchesdoesrules[rule_id].shortdesc→KeyError: 'E0003'main()has noRunnerwhen construction fails, so it has no rules. Instead,__init__records the message,cli()reports it through the unchanged_cli_output, andrun()raises. That keeps the CLI byte-identical by construction instead of by re-implementation, and needs no refactor of the shared exit path.Which exception.
ConfigFileErroralready exists and carries a config for formatting, but it is not a match: its exit code is 1 where this path's is 2, and its docstring scopes it to the.cfnlintrcfile, while this is a CLI argument that resolves to no files. Reusing it would have changed the exit code for every format.TemplatesNotFoundErroris aCfnLintExitExceptionwith exit code 2, matching what the CLI already returns.2.
__all__was missing three of five subclasses.cfnlint.runnerexportedCfnLintExitException,InvalidRegionExceptionandUnexpectedRuleException, but notDuplicateRuleError,ConfigFileErrororSchemaPatchError. All are library-facing and that module is where a caller looks for them. All five are now exported, and a test derives the expected set fromcfnlint.exceptionsso a future subclass cannot be added without being exported.CLI contract
python -c 'from cfnlint.runner import main; main()' --format <fmt> -t '<unresolvable glob>', diffed byte-for-byte againstupstream/main:--format parseable--format json--format junit--format sarif--format quiet--format pretty--non-zero-exit-code none--output-fileControls — a clean template, a template with a finding, and
--list-rules— are identical in exit code and bytes.The one deliberate difference.
--format prettyon an unresolvable glob crashes with an uncaughtValueErroronmainand on this branch:PrettyFormatter.print_matchesreadsconfig.templatesagain (formatters/pretty.py:32), which raises the same error a second time. Exit code (1), stdout (empty) and the final exception are unchanged; only the traceback text differs, because the error is no longer raised from inside anexceptblock so there is no "During handling of the above exception" chain. This is a pre-existing bug inPrettyFormatterand is left alone rather than widening this PR.Testing
The 6 failures are the missing optional dependencies (
pydot,junit-xml,sarif) and are the same set asupstream/main(6 failed, 3269 passed).test_no_templatesasserted the old behaviour — thatRunner(config)itself prints and exits — withsys.exitpatched to a no-op, which is what made the process kill look survivable. It now asserts that construction is silent, and thatcli()produces the same bytes and the real exit code 2.Each test was mutation-checked by reverting the change it pins:
runner/cli.py→ 6 tests fail,test_no_templateswithSystemExit: 2runner/__init__.py__all__→ 5 failuresSchemaPatchErrorfrom__all__→ still 2 failures, so the derived test is not vacuousruff check,ruff format --check,isort,mypyandpre-commit run --all-filesall pass.sys.exitauditgrep -rn 'sys\.exit' src/cfnlint/returns 14 calls, all inrunner/cli.py: 13 directly incli()ormain(), and one inRunner._exit, which is reached only through_cli_output, which after this change is called only fromcli(). None is reachable from a constructor or from anycfnlint.apifunction.api.pynever callscli().By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.