Skip to content

Warn about the DatabricksSQLStatementsSensor exclusivity tightening - #72761

Merged
eladkal merged 1 commit into
apache:mainfrom
moomindani:databricks-changelog-sensor-exclusivity-warning
Sep 9, 2026
Merged

eladkal merged 1 commit into
apache:mainfrom
moomindani:databricks-changelog-sensor-exclusivity-warning

Conversation

@moomindani

Copy link
Copy Markdown
Contributor

#70831 moved DatabricksSQLStatementsSensor's exclusivity check from execute() to __init__ and changed it from a truthiness test to is not None. That is a behaviour change against a released version, because providers-databricks/7.19.0 (tagged 2026-08-19) already contains #70340 and so ships the execute()-time truthiness check:

  • statement="SELECT 1" with statement_id="" runs the statement in 7.19.0, and is a Dag import error now.
  • Same for a statement_id template that renders to None under render_template_as_native_obj=True — it is a non-None string at construction time, so the pair is rejected at parse time.
  • The two argument-validation errors are now ValueError, so an except AirflowException around them stops matching.

The tightening itself is sanctioned by #70503 and is not in question here. The note explaining it to users was requested during the review of #70831 but did not land before the merge, so the Changelog header in this provider still has nothing under it.

The wording is the text written in that review, placed where the file's own header comment asks for it (just below Changelog, above the topmost version), in the same shape as the ComprehendCreateDocumentClassifierOperator warning at the top of the amazon changelog.

I verified both halves before writing it: providers-databricks/7.19.0's sensor raises AirflowException from execute() under if self.statement and self.statement_id, and main's raises ValueError from __init__ under if statement is not None and statement_id is not None. Providers take no newsfragment, so the changelog is the only place this can be said. check-changelog-format passes.

related: #70831


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

The sensor's exclusivity check moved to Dag parse time and now treats an empty
string as a provided value, so a pairing that ran the statement in the released
7.19.0 fails at parse time instead. The note was asked for while apache#70831 was in
review and the PR merged before it landed, leaving users upgrading past 7.19.0
with no explanation of the failure.
@moomindani

Copy link
Copy Markdown
Contributor Author

@potiuk this is the changelog note from your review of #70831, placed where the file's header comment asks for it. The wording is yours, unchanged.


Drafted-by: Claude Code (Opus 5); reviewed by @moomindani before posting

@eladkal
eladkal merged commit ac03ede into apache:main Sep 9, 2026
70 checks passed
imrichardwu pushed a commit to imrichardwu/airflow that referenced this pull request Sep 11, 2026
…pache#72761)

The sensor's exclusivity check moved to Dag parse time and now treats an empty
string as a provided value, so a pairing that ran the statement in the released
7.19.0 fails at parse time instead. The note was asked for while apache#70831 was in
review and the PR merged before it landed, leaving users upgrading past 7.19.0
with no explanation of the failure.
xvega pushed a commit to xvega/airflow that referenced this pull request Sep 13, 2026
…pache#72761)

The sensor's exclusivity check moved to Dag parse time and now treats an empty
string as a provided value, so a pairing that ran the statement in the released
7.19.0 fails at parse time instead. The note was asked for while apache#70831 was in
review and the PR merged before it landed, leaving users upgrading past 7.19.0
with no explanation of the failure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants