Fix template-field validation timing in BigQueryToMsSqlOperator - #70678
Fix template-field validation timing in BigQueryToMsSqlOperator#70678bramhanandlingala wants to merge 2 commits into
Conversation
|
Thanks for the work here, but this duplicates #70493, which makes the same change to the same three files and was opened two days earlier. That one has already been reviewed and approved, so I'm closing this in its favour. This is the third time this week a PR of yours has landed on top of someone else's earlier one — #70621 duplicated #70449, and #70437 duplicates #70369. I'm not questioning the quality of the work; the analysis in all three was sound. But each duplicate costs the original author a rebase race and a reviewer a second pass over the same diff, and it's avoidable. Before starting on an operator, There are still several unclaimed entries in Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting |
Summary
BigQueryToMsSqlOperatorparsed its templatedsource_project_dataset_tablefield inside
__init__, before Jinja rendering runs. Fixes this entry fromthe #70296 exemption-list burn-down.
Root cause
source_project_dataset_tableis listed intemplate_fields, but__init__called
.split(".")on it immediately to derivedataset_id/table_id.Template fields are only rendered after the constructor returns, so this
ran against the raw, un-rendered string — a templated value like
"{{ params.project }}.{{ params.dataset }}.{{ params.table }}"eitherfails to parse or produces garbage components.
Fix
__init__now does a plain assignment ofsource_project_dataset_tableand initializes
dataset_id/table_idtoNone..split(".")parsing moved toexecute(), where it runs against therendered value before delegating to
super().execute(context).mssql_tablevs.target_table_nameprovision check wasleft untouched in
__init__, since it only checks whether an argumentwas supplied, not its content.
Related to #70296
Gen-AI disclosure: I used a generative AI tool to help identify the root
cause, write tests, and draft the PR description. I reviewed, tested, and
verified all changes locally before submitting.
Generated-by: Claude following the guidelines