Skip to content

docs: clarify Redis maintenance for CeleryExecutor - #67393

Merged
potiuk merged 1 commit into
apache:mainfrom
Will-thom:issue-55215-redis-maintenance-docs
Jul 4, 2026
Merged

docs: clarify Redis maintenance for CeleryExecutor#67393
potiuk merged 1 commit into
apache:mainfrom
Will-thom:issue-55215-redis-maintenance-docs

Conversation

@Will-thom

Copy link
Copy Markdown
Contributor

Closes #55215.\n\nAdds a Redis maintenance section to the CeleryExecutor docs explaining what Airflow stores in Redis when it is used as the Celery broker, how that differs from the metadata database, when cleanup is appropriate, and why Redis should not be flushed while schedulers or workers are running.\n\nValidation:\n- git diff --check -- providers/celery/docs/celery_executor.rst\n- rg -n Redis maintenance providers/celery/docs/celery_executor.rst

@kaxil

kaxil commented May 24, 2026

Copy link
Copy Markdown
Member

What's the motivation behind adding this?

@kaxil kaxil added pending-response and removed ready for maintainer review Set after triaging when all criteria pass. labels May 24, 2026
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 28, 2026
@Will-thom

Copy link
Copy Markdown
Contributor Author

The motivation is to make the Redis maintenance boundary explicit for CeleryExecutor users.

The linked issue asks what Airflow stores in Redis and whether it is safe to clean it. Today the docs explain Redis as the broker, but they do not clearly distinguish broker-side transient Celery state from the Airflow metadata database, nor warn that flushing Redis while schedulers/workers are active can drop queued work or make task state inconsistent.

This section is meant to prevent operators from treating Redis cleanup like airflow db clean: metadata cleanup and broker cleanup affect different stores and need different safety conditions.

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because the author has not responded to a request for more information. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Jun 11, 2026
@eladkal eladkal removed stale Stale PRs per the .github/workflows/stale.yml policy file pending-response labels Jun 11, 2026
@eladkal
eladkal requested a review from kaxil June 11, 2026 05:34

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs-only — accurate Redis-broker maintenance guidance for the Celery executor. Verified the diff. LGTM.

@potiuk
potiuk merged commit 34604d2 into apache:main Jul 4, 2026
70 checks passed
@boring-cyborg

boring-cyborg Bot commented Jul 4, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Not clear when and what to clean on Redis

4 participants