Allow providers to ship testing Dags separately from example Dags - #70541
Open
ColtenOuO wants to merge 1 commit into
Open
Allow providers to ship testing Dags separately from example Dags#70541ColtenOuO wants to merge 1 commit into
ColtenOuO wants to merge 1 commit into
Conversation
Example Dags currently serve four different audiences at once: they teach Dag authoring, they back documentation snippets, they verify a deployment, and some of them are load-bearing for CI. Because everything lives in one folder, a user who enables `[core] load_examples` gets Dags that only exist to exercise Airflow itself -- `example_failed_dag`, for instance, is designed to fail -- and nobody can safely delete a stale example without first working out whether a test depends on it. This adds the loading mechanism for a separate `testing_dags` folder per provider so the two purposes can be told apart. No Dags move yet; that happens per provider in follow-ups. Part of apache#52475.
ColtenOuO
requested review from
amoghrajesh,
ashb,
bugraoz93,
choo121600,
ephraimbuddy,
gopidesupavan,
jason810496,
jedcunningham,
jscheffl,
potiuk and
vatsrahul1001
as code owners
July 27, 2026 15:53
Contributor
Author
|
Just a couple more questions before we wrap this up!
|
jason810496
reviewed
Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the loading mechanism for a per-provider
testing_dags/folder, so Dags thatexist to exercise Airflow itself can be told apart from Dags that teach Dag authoring.
No Dags move in this PR. Moving them happens one provider at a time in follow-ups.
Why
Example Dags currently serve four audiences at once: teaching Dag authoring, backing
documentation snippets, verifying a deployment, and propping up CI. Because they all
share one folder:
[core] load_examplesalso gets Dags that only exist for testing—
example_failed_dagis designed to fail;on it. Today
example_failed_dag,example_skip_dag,example_passing_params_via_test_commandandexample_kubernetes_executorarereferenced from
airflow-core/tests/unit/cli/,airflow-core/tests/unit/api_fastapi/,airflow-e2e-tests/andkubernetes-tests/.What's in here
[core] load_testing_dagsconfig option, defaulting toFalseand independent ofload_examples.DagBundlesManagerdiscovers<provider>/testing_dags/and exposes it as anapache-airflow-providers-*-testing-dagsbundle. The provider-resolution loop added inLoad example DAGs from providers via ProvidersManager (continuation of #57320) #66161 is factored out into
_iter_provider_module_paths()so both folder kinds shareit.
breeze shell/breeze start-airflow --load-testing-dags, plumbed through toAIRFLOW__CORE__LOAD_TESTING_DAGS.load_provider_testing_dagspytest fixture intests_common, so the follow-up PRs canpull these Dags into unit tests from both breeze and a local venv.
contributing-docs/12_provider_distributions.rst.Since no provider ships a
testing_dags/folder yet, this is inert for existingdeployments.
related: #52475
Was generative AI tooling used to co-author this PR?