Skip to content

fix: add ref() macro override for dbt-core 1.11 compatibility#2137

Merged
haritamar merged 2 commits intomasterfrom
devin/1772470747-dbt-core-1.11-ref-syntax
Mar 2, 2026
Merged

fix: add ref() macro override for dbt-core 1.11 compatibility#2137
haritamar merged 2 commits intomasterfrom
devin/1772470747-dbt-core-1.11-ref-syntax

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot commented Mar 2, 2026

Summary

Adds a ref() Jinja macro override in the elementary CLI's internal monitor dbt project (elementary_cli) to fix cross-package ref() calls breaking on adapters that override ref() with an incompatible signature (specifically dbt-dremio).

Root cause: dbt-dremio overrides ref() with ref(model_name, v=None), which intercepts the second positional arg as a version identifier. When the monitor project calls ref('elementary', 'dbt_models'), dbt-dremio treats it as builtins.ref('elementary', v='dbt_models') — looking for a model named "elementary" at version "dbt_models" — instead of a cross-package ref. This is a known upstream bug: dremio/dbt-dremio#232 (open since Jun 2024, unresolved).

Fix: A project-level ref() macro that properly dispatches all argument forms directly to builtins.ref(), which correctly handles two-arg positional cross-package refs in dbt-core 1.11+. Project macros take precedence over adapter macros in dbt's resolution order, so this overrides dbt-dremio's broken version.

The macro handles all ref() forms including combined cross-package + versioned refs (e.g. ref('package', 'model', v=1)) for forward compatibility, using an effective_version consolidation approach.

Review & Testing Checklist for Human

  • Verify project-level ref() actually overrides adapter-level ref(): dbt's macro resolution gives priority to root project macros over adapter macros, but ref is a special builtin — confirm this override takes effect when running with dbt-dremio (and other adapters). If it doesn't, this PR has no effect.
  • Verify no adapter relies on ref() override for critical functionality that we'd be bypassing: dbt-dremio's ref() adds format/reflection handling. For the monitor project (which only reads existing tables), this should be safe to bypass — but confirm there's no case where the monitor project creates/materializes models that need adapter-specific ref behavior.
  • Test with dbt-dremio + dbt-core 1.11: Run edr monitor or edr report against a Dremio instance to confirm cross-package refs like ref('elementary', 'dbt_models') and self-refs like ref('elementary_cli', 'alerts_v2') resolve correctly.
  • Test with another adapter (e.g. Postgres/Snowflake): Confirm the override doesn't break normal ref() behavior on adapters that don't override ref().

Suggested test plan: Run the Dremio CI job from PR #2135 without continue-on-error on the edr steps, using this branch's monitor project. If edr monitor report succeeds, the fix works.

Notes

  • This is an elementary-side workaround for an upstream dbt-dremio bug (dremio/dbt-dremio#232). The proper fix would be in dbt-dremio's ref macro to support (*args, **kwargs) and forward them correctly to builtins. dbt-dremio is currently the only adapter that overrides ref() this way.
  • The macro supports the legacy package= keyword form (ref('model', package='pkg')) for backward compatibility with older dbt-core versions, though dbt-core 1.11 itself removed this.
  • Risk: This macro override applies to ALL adapters when running the monitor project, not just dbt-dremio. If any adapter's ref() override performs critical operations (like Dremio's format/reflection handling), those operations are bypassed. For a read-only monitor project this should be safe, but worth double-checking.
  • CI warehouse test failures (Snowflake, Redshift, BigQuery, Databricks, Athena) are pre-existing numeric_column_anomalies + error_model issues on master, unrelated to this change.

Link to Devin Session: https://app.devin.ai/sessions/6c12a19d456541568bcd7974623f3557
Requested by: @haritamar

Summary by CodeRabbit

  • New Features
    • Expanded ref() handling to accept simple, cross-package, and versioned reference forms (including legacy keyword variants).
    • Added fallback routing so all supported ref() call formats behave consistently.
    • Ensures compatibility with dbt-core 1.11 and later.

Some adapters (e.g. dbt-dremio) override the ref() Jinja macro with a
signature that doesn't support the two-argument positional form used for
cross-package references: ref('package', 'model').

dbt-dremio's ref(model_name, v=None) intercepts the second positional
arg as a version identifier, breaking refs like ref('elementary', 'dbt_models').

This override in the elementary_cli monitor project forwards all arguments
directly to builtins.ref(), which correctly handles all ref() forms in
dbt-core 1.11+. It also supports the legacy package= keyword form for
backward compatibility.

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 2, 2026

👋 @devin-ai-integration[bot]
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in this pull request.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 2, 2026

📝 Walkthrough

Walkthrough

Adds a new DBT macro override ref() that normalizes and forwards various call signatures (simple, cross-package, and versioned references) to builtins.ref() for consistent resolution across dbt-core 1.11+.

Changes

Cohort / File(s) Summary
New DBT ref() macro override
elementary/monitor/dbt_project/macros/overrides/ref.sql
Adds ref(model_name_or_package, model_name=none, v=none, version=none, package=none) macro that detects call patterns—ref('model'), ref('package','model'), ref('model', v=...), ref('model', version=...), ref('model', package='...')—and delegates to builtins.ref() accordingly, with a final single-argument fallback.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I hop through macros, tidy each ref call,
Package names, versions—I'll catch them all.
Builtins await where I gently steer,
Models resolved, the path made clear.
A little rabbit keeps dbt in thrall.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: adding a ref() macro override specifically to address dbt-core 1.11 compatibility issues caused by adapter incompatibilities.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch devin/1772470747-dbt-core-1.11-ref-syntax

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@elementary/monitor/dbt_project/macros/overrides/ref.sql`:
- Around line 17-34: The ref macro currently branches on model_name/package
before handling version, which drops v/version for combined
cross-package+version calls; update the macro ref(model_name_or_package,
model_name=none, v=none, version=none, package=none) to first compute an
effective_version (e.g., effective_version = v if v is not none else version)
and then pass that effective_version into every builtins.ref call that takes two
arguments (both the two-arg positional branch using
builtins.ref(model_name_or_package, model_name) and the legacy package branch
using builtins.ref(package, model_name_or_package)), and also use
builtins.ref(model_name_or_package, v=effective_version) for single-arg
versioned calls so that any combination of cross-package + version preserves the
version argument.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 746ce41 and f91c584.

📒 Files selected for processing (1)
  • elementary/monitor/dbt_project/macros/overrides/ref.sql

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

Note: This PR is a workaround for an open upstream bug in dbt-dremio: dremio/dbt-dremio#232 — Override of ref macro does not support packages or the "version" keyword argument (opened Jun 2024, still unresolved).

dbt-dremio's ref(model_name, v=None) override doesn't handle the two-positional-argument cross-package form ref('package', 'model') or the version keyword — it incorrectly interprets the second positional arg as a version identifier. This breaks any dbt project that uses cross-package refs with dbt-dremio + dbt-core 1.11+.

Our project-level override properly forwards all argument forms to builtins.ref(), bypassing dbt-dremio's broken macro. dbt-dremio is currently the only adapter that overrides ref() this way.

…tibility

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
elementary/monitor/dbt_project/macros/overrides/ref.sql (1)

17-40: Add a regression test matrix for this override path.

Consider adding an integration/macro test that exercises ref('pkg','model'), ref('model', package='pkg'), and their versioned variants. This will lock in the compatibility fix and prevent future regressions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/monitor/dbt_project/macros/overrides/ref.sql` around lines 17 -
40, The ref override macro handles multiple call patterns (macro ref, variable
effective_version, and calls to builtins.ref) and needs coverage to prevent
regressions; add an integration/macro test matrix that calls the two-arg
positional form ref('pkg','model'), the legacy keyword form ref('model',
package='pkg'), the single-arg versioned forms ref('model', v=1') and
ref('model', version=1), plus the two-arg versioned variant ref('pkg','model',
v=1) (or equivalent), asserting that each resolves to the same target and that
builtins.ref is invoked with the expected arguments for each branch (including
branches that use effective_version and package/model_name). Ensure tests
exercise the branches guarded by model_name, package, and effective_version so
future changes to macro ref or builtins.ref calls will be detected.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@elementary/monitor/dbt_project/macros/overrides/ref.sql`:
- Around line 17-40: The ref override macro handles multiple call patterns
(macro ref, variable effective_version, and calls to builtins.ref) and needs
coverage to prevent regressions; add an integration/macro test matrix that calls
the two-arg positional form ref('pkg','model'), the legacy keyword form
ref('model', package='pkg'), the single-arg versioned forms ref('model', v=1')
and ref('model', version=1), plus the two-arg versioned variant
ref('pkg','model', v=1) (or equivalent), asserting that each resolves to the
same target and that builtins.ref is invoked with the expected arguments for
each branch (including branches that use effective_version and
package/model_name). Ensure tests exercise the branches guarded by model_name,
package, and effective_version so future changes to macro ref or builtins.ref
calls will be detected.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between f91c584 and 25bd3ba.

📒 Files selected for processing (1)
  • elementary/monitor/dbt_project/macros/overrides/ref.sql

@haritamar haritamar merged commit 404708b into master Mar 2, 2026
11 of 16 checks passed
@haritamar haritamar deleted the devin/1772470747-dbt-core-1.11-ref-syntax branch March 2, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant