Skip to content

Fix template-field validation timing in BigQueryToMsSqlOperator - #70678

Closed
bramhanandlingala wants to merge 2 commits into
apache:mainfrom
bramhanandlingala:fix/70296-8
Closed

Fix template-field validation timing in BigQueryToMsSqlOperator#70678
bramhanandlingala wants to merge 2 commits into
apache:mainfrom
bramhanandlingala:fix/70296-8

Conversation

@bramhanandlingala

Copy link
Copy Markdown
Contributor

Summary
BigQueryToMsSqlOperator parsed its templated source_project_dataset_table
field inside __init__, before Jinja rendering runs. Fixes this entry from
the #70296 exemption-list burn-down.

Root cause
source_project_dataset_table is listed in template_fields, but __init__
called .split(".") on it immediately to derive dataset_id/table_id.
Template fields are only rendered after the constructor returns, so this
ran against the raw, un-rendered string — a templated value like
"{{ params.project }}.{{ params.dataset }}.{{ params.table }}" either
fails to parse or produces garbage components.

Fix

  • __init__ now does a plain assignment of source_project_dataset_table
    and initializes dataset_id/table_id to None.
  • The .split(".") parsing moved to execute(), where it runs against the
    rendered value before delegating to super().execute(context).
  • The existing mssql_table vs. target_table_name provision check was
    left untouched in __init__, since it only checks whether an argument
    was supplied, not its content.

Related to #70296

Gen-AI disclosure: I used a generative AI tool to help identify the root
cause, write tests, and draft the PR description. I reviewed, tested, and
verified all changes locally before submitting.

  • Yes - Claude

Generated-by: Claude following the guidelines

@potiuk

potiuk commented Jul 30, 2026

Copy link
Copy Markdown
Member

Thanks for the work here, but this duplicates #70493, which makes the same change to the same three files and was opened two days earlier. That one has already been reviewed and approved, so I'm closing this in its favour.

This is the third time this week a PR of yours has landed on top of someone else's earlier one — #70621 duplicated #70449, and #70437 duplicates #70369. I'm not questioning the quality of the work; the analysis in all three was sound. But each duplicate costs the original author a rebase race and a reviewer a second pass over the same diff, and it's avoidable.

Before starting on an operator, gh pr list --search "<operator name>" (or a quick search on the PR list) will show whether one is already in flight. If there is, reviewing it or offering an improvement in a comment is far more valuable than a parallel PR — Airflow's "better PR wins" convention exists for genuinely different approaches, not near-identical ones.

There are still several unclaimed entries in scripts/ci/prek/validate_operators_init_exemptions.txt if you'd like to pick one up — just worth checking first that nobody else has.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools area:providers backport-to-v3-3-test Backport to v3-3-test provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants