Skip to content

Breeze: Make postgres driver explicit and version-aware - #70496

Merged
potiuk merged 2 commits into
apache:mainfrom
shahar1:fix-breeze-postgres-scheme
Jul 29, 2026
Merged

Breeze: Make postgres driver explicit and version-aware#70496
potiuk merged 2 commits into
apache:mainfrom
shahar1:fix-breeze-postgres-scheme

Conversation

@shahar1

@shahar1 shahar1 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Since #69469 moved Breeze's backend-postgres.yml to a bare postgresql:// scheme (so that core's autodetection selects psycopg3), every Breeze postgres session running Airflow from sources emits a FutureWarning at config load — main now lists the bare scheme in bad_schemes.

Making the driver explicit has to be version-aware: the migration tests install a released Airflow into the same container, no released version ships psycopg (v3) in its default constraints, and releases before 3.2.0 can run on SQLAlchemy 1.4, which has no postgresql+psycopg dialect at all — an explicit psycopg3 URL crashes them with sqlalchemy.exc.NoSuchModuleError (this is exactly what failed the Postgres core jobs on this PR's first revision).

Breeze now derives POSTGRES_DRIVER from --use-airflow-version — psycopg2 when a released version is installed, psycopg for sources/wheel/sdist/branch/PR installs — and the compose file interpolates it into both sql_alchemy_conn and the Celery result backend URL, defaulting to psycopg.

Verified locally: the Airflow 2.11.0 migration leg resolves postgresql+psycopg2:// and completes, and a from-sources session resolves postgresql+psycopg:// with no FutureWarning.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Fable 5, Opus 5)

Generated-by: Claude Code (Fable 5, Opus 5) following the guidelines

@Dev-iL Dev-iL left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't look like this setting has any backward compatibility concerns, so the fix should be fine.

I wonder why this was skipped in #69469. Perhaps it was a matter of ordering...

@potiuk

potiuk commented Jul 28, 2026

Copy link
Copy Markdown
Member

There are some errors to fix ?

@potiuk

potiuk commented Jul 28, 2026

Copy link
Copy Markdown
Member

@Dev-iL - maybe you can take a look ?

@Dev-iL

Dev-iL commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Ok so here's the deal:

  • The warning fires on 2.11.0 which ships an older version of airflow-core/src/airflow/configuration.py that predates the psycopg v3 autodetection.
  • It should be possible fix the old-airflow breeze shell invocations by specifying --custom-db-url as "postgresql://postgres:airflow@postgres/airflow"

@shahar1
shahar1 force-pushed the fix-breeze-postgres-scheme branch from 728457d to c3eb6d1 Compare July 28, 2026 12:40
@Dev-iL

Dev-iL commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

@shahar1 @potiuk merge?

@shahar1

shahar1 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

@shahar1 @potiuk merge?

I will merge once approved, still investigating the root cause of the issue :)

shahar1 added 2 commits July 28, 2026 18:38
…es backend

apache#69469 moved Breeze's compose file to a bare `postgresql://` so core's
autodetection would select psycopg3. On main that scheme is in `bad_schemes`,
so a Breeze session running from sources now emits a FutureWarning at config
load. Name the driver explicitly instead, matching the celery result backend
URL one line below.

Generated-by: Claude Code (Fable 5)
Making the driver explicit silences the FutureWarning that a Breeze postgres
session running from sources emits, but migration tests install a released
Airflow into the same container, and releases before 3.2.0 can run on
SQLAlchemy 1.4, which has no postgresql+psycopg dialect at all. Selecting the
driver per installed Airflow keeps runs from sources on psycopg3, which main
selects by default, without breaking the legs that install a release.
@shahar1
shahar1 force-pushed the fix-breeze-postgres-scheme branch from c3eb6d1 to 5b628d3 Compare July 28, 2026 15:39
@Dev-iL

Dev-iL commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Huh, CI was green earlier

@shahar1

shahar1 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Huh, CI was green earlier

Huh, CI was green earlier

Unrelated to this PR, main is currently broken. Trying to work on a fix :)

@shahar1 shahar1 changed the title Breeze: Make postgres driver explicit in sql_alchemy_conn to silence FutureWarning Breeze: Make postgres driver explicit and version-aware Jul 28, 2026
@potiuk
potiuk merged commit 221da63 into apache:main Jul 29, 2026
152 of 153 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-3-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-3-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 221da63 v3-3-test

This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants