Skip to content

Add FAB migration commands - #41804

Merged
ephraimbuddy merged 17 commits into
apache:mainfrom
astronomer:add-fab-upgrade-commands
Sep 19, 2024
Merged

Add FAB migration commands#41804
ephraimbuddy merged 17 commits into
apache:mainfrom
astronomer:add-fab-upgrade-commands

Conversation

@ephraimbuddy

Copy link
Copy Markdown
Contributor

This PR adds migrate, upgrade and reset db commands to facilitate migrating FAB DBs.

FAB upgrade is also integrated into Airflow upgrade such that if airflow db is being upgraded to the heads, FAB migration will also upgrade to the heads.

Migration checks to determine if migration has finished now includes checking that FAB migration is also done.

Note that downgrading Airflow does not trigger FAB downgrade. FAB downgrade has to be done with FAB downgrade command.

@ephraimbuddy
ephraimbuddy marked this pull request as draft August 27, 2024 18:08
@ephraimbuddy
ephraimbuddy force-pushed the add-fab-upgrade-commands branch 4 times, most recently from f7e2b2c to 0ef502a Compare August 29, 2024 10:19
@ephraimbuddy
ephraimbuddy marked this pull request as ready for review August 29, 2024 10:21
@ephraimbuddy
ephraimbuddy force-pushed the add-fab-upgrade-commands branch from 0ef502a to 416a964 Compare August 29, 2024 13:07

@shahar1 shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, my main conerns are regarding having both changes to core and providers.

Comment thread airflow/cli/commands/db_command.py Outdated
Comment thread airflow/cli/commands/db_command.py Outdated
Comment thread airflow/providers/fab/auth_manager/cli_commands/db_command.py Outdated
Comment thread airflow/providers/fab/auth_manager/fab_auth_manager.py Outdated
@ephraimbuddy
ephraimbuddy force-pushed the add-fab-upgrade-commands branch from 416a964 to f1915b4 Compare August 30, 2024 10:10
@shahar1 shahar1 added this to the Airflow 3.0.0 milestone Aug 30, 2024
@ephraimbuddy
ephraimbuddy force-pushed the add-fab-upgrade-commands branch 6 times, most recently from 6ea9c9d to e4d4bdd Compare September 4, 2024 21:13
@ephraimbuddy
ephraimbuddy force-pushed the add-fab-upgrade-commands branch from e4d4bdd to b615aae Compare September 17, 2024 11:14
Comment thread airflow/cli/commands/db_command.py Outdated
Comment thread airflow/providers/fab/migrations/versions/0001_1_3_0_placeholder_migration.py Outdated
@ephraimbuddy
ephraimbuddy force-pushed the add-fab-upgrade-commands branch from 2baf5a1 to 9242359 Compare September 18, 2024 09:44
Comment thread airflow/cli/commands/db_command.py Outdated
This PR adds `migrate`, `upgrade` and `reset` db commands to facilitate
migrating FAB DBs.

FAB upgrade is also integrated into Airflow upgrade such that if airflow
db is being upgraded to the heads, FAB migration will also upgrade to the heads.

Migration checks to determine if migration has finished now includes checking that
FAB migration is also done.

Note that downgrading Airflow does not trigger FAB downgrade. FAB downgrade has to
be done with FAB downgrade command.
@ephraimbuddy
ephraimbuddy force-pushed the add-fab-upgrade-commands branch from 583dfee to 4c6cea7 Compare September 18, 2024 23:37
@ephraimbuddy
ephraimbuddy merged commit 9f167bb into apache:main Sep 19, 2024
@ephraimbuddy
ephraimbuddy deleted the add-fab-upgrade-commands branch September 19, 2024 07:21
joaopamaral pushed a commit to joaopamaral/airflow that referenced this pull request Oct 21, 2024
* Add FAB migration commands

This PR adds `migrate`, `upgrade` and `reset` db commands to facilitate
migrating FAB DBs.

FAB upgrade is also integrated into Airflow upgrade such that if airflow
db is being upgraded to the heads, FAB migration will also upgrade to the heads.

Migration checks to determine if migration has finished now includes checking that
FAB migration is also done.

Note that downgrading Airflow does not trigger FAB downgrade. FAB downgrade has to
be done with FAB downgrade command.

* Fix failing tests

* Add tests

* add more tests

* fixup! add more tests

* fixup! fixup! add more tests

* Refactor code

* fixup! Refactor code

* set airflow_db as false

* Add placeholder migration for correct stamping

* fixup! Add placeholder migration for correct stamping

* fixup! fixup! Add placeholder migration for correct stamping

* fix test

* fixup! fix test

* fix rebase mistake

* use reserialize_dags instead of airflow_db

* add messages around placeholder migration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants