Skip to content

Remove hard-coded deferrable crawler run from example_glue system test - #70206

Merged
o-nikolas merged 5 commits into
apache:mainfrom
aws-mwaa:fix/example-glue-deferrable-hang
Jul 23, 2026
Merged

Remove hard-coded deferrable crawler run from example_glue system test#70206
o-nikolas merged 5 commits into
apache:mainfrom
aws-mwaa:fix/example-glue-deferrable-hang

Conversation

@seanghaeli

Copy link
Copy Markdown
Contributor

The run_crawler_deferrable task added in #69930 hard-codes deferrable=True, which hangs forever under dag.test(use_executor=True) (no triggerer runs in that path), timing out executor-based system test harnesses. The non-deferrable run_crawler task covers the operator; deferrable behavior is exercised by unit tests and the harness-level deferrable mode (AIRFLOW__OPERATORS__DEFAULT_DEFERRABLE).

@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Jul 22, 2026
@seanghaeli seanghaeli closed this Jul 22, 2026
@seanghaeli seanghaeli reopened this Jul 22, 2026
@seanghaeli
seanghaeli marked this pull request as ready for review July 22, 2026 20:59
@seanghaeli
seanghaeli requested a review from o-nikolas as a code owner July 22, 2026 20:59
Comment thread providers/amazon/tests/system/amazon/aws/example_glue.py Outdated

@AlejandroMorgante AlejandroMorgante left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, if we remove run_crawler_deferrable, we should update providers/amazon/docs/operators/glue.rst, since it still references the removed howto_operator_glue_crawler_run_deferrable markers through an exampleinclude.

@o-nikolas
o-nikolas merged commit 8004f48 into apache:main Jul 23, 2026
83 checks passed
@o-nikolas
o-nikolas deleted the fix/example-glue-deferrable-hang branch July 23, 2026 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants