[v3-3-test] Hide team scoped values of options registered as sensitive (#70755) - #71099
Merged
Conversation
#70755) * Hide team scoped values of options registered as sensitive Options are registered as sensitive under their base section, but a team scoped override lives in a `[<team>=<section>]` config file section, or in an `AIRFLOW__<TEAM>___<SECTION>__<KEY>` environment variable. Every sensitivity decision was a direct membership test against the registered base pairs, and the masking pass iterated those pairs and looked each section up verbatim, so a team scoped section was never visited and its value was returned in clear. Resolve the team scoped spelling back to the base option before deciding whether a value is sensitive. `team_section_name` now builds the config file section name at both construction sites so the two representations cannot drift, `base_section_name` recovers the base section from it, and `is_sensitive_option` tests a pair directly, then via the base section, then via the tail an environment variable contributes -- so it can only ever recognise more options as sensitive, never fewer. The team name is not parsed out of an environment variable name, because a team name may contain underscores; the name is matched against the tail each registered option contributes instead. The config file section name is split on the last separator, so the base section is recovered even for a team name that contains the separator itself. Four call sites use the predicate: the masking pass, the environment collection, `write`, and the single option config route, which does not go through `as_dict` and so needed its own change. Team scoped `_cmd` and `_secret` entries are hidden in place rather than resolved into their value, because resolving them is not supported for a team. Generated-by: Claude Opus 5 (1M context) following the guidelines at https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions * Add newsfragment for the team scoped sensitive option masking change * Keep one copy of the team scoped masking rationale, and record the untouched gap The per-key secrets-backend options are matched by literal section name and stay unaware of a team scoped spelling. Nothing leaks while the secrets backend itself is not team aware, so it is recorded rather than fixed here. * Update shared/configuration/tests/configuration/test_parser.py Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com> * Update airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_config.py Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com> --------- (cherry picked from commit d41ac7b) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
5 tasks
vatsrahul1001
marked this pull request as ready for review
August 4, 2026 14:01
vatsrahul1001
requested review from
amoghrajesh,
ashb,
bugraoz93,
choo121600,
ephraimbuddy,
henry3260,
jason810496,
pierrejeambrun,
potiuk,
rawwar and
shubhamraj-git
as code owners
August 4, 2026 14:01
vatsrahul1001
pushed a commit
that referenced
this pull request
Aug 5, 2026
#70755) (#71099) * Hide team scoped values of options registered as sensitive Options are registered as sensitive under their base section, but a team scoped override lives in a `[<team>=<section>]` config file section, or in an `AIRFLOW__<TEAM>___<SECTION>__<KEY>` environment variable. Every sensitivity decision was a direct membership test against the registered base pairs, and the masking pass iterated those pairs and looked each section up verbatim, so a team scoped section was never visited and its value was returned in clear. Resolve the team scoped spelling back to the base option before deciding whether a value is sensitive. `team_section_name` now builds the config file section name at both construction sites so the two representations cannot drift, `base_section_name` recovers the base section from it, and `is_sensitive_option` tests a pair directly, then via the base section, then via the tail an environment variable contributes -- so it can only ever recognise more options as sensitive, never fewer. The team name is not parsed out of an environment variable name, because a team name may contain underscores; the name is matched against the tail each registered option contributes instead. The config file section name is split on the last separator, so the base section is recovered even for a team name that contains the separator itself. Four call sites use the predicate: the masking pass, the environment collection, `write`, and the single option config route, which does not go through `as_dict` and so needed its own change. Team scoped `_cmd` and `_secret` entries are hidden in place rather than resolved into their value, because resolving them is not supported for a team. Generated-by: Claude Opus 5 (1M context) following the guidelines at https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions * Add newsfragment for the team scoped sensitive option masking change * Keep one copy of the team scoped masking rationale, and record the untouched gap The per-key secrets-backend options are matched by literal section name and stay unaware of a team scoped spelling. Nothing leaks while the secrets backend itself is not team aware, so it is recorded rather than fixed here. * Update shared/configuration/tests/configuration/test_parser.py * Update airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_config.py --------- (cherry picked from commit d41ac7b) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Options are registered as sensitive under their base section, but a team scoped
override lives in a
[<team>=<section>]config file section, or in anAIRFLOW__<TEAM>___<SECTION>__<KEY>environment variable. Every sensitivitydecision was a direct membership test against the registered base pairs, and the
masking pass iterated those pairs and looked each section up verbatim, so a team
scoped section was never visited and its value was returned in clear.
Resolve the team scoped spelling back to the base option before deciding whether
a value is sensitive.
team_section_namenow builds the config file section nameat both construction sites so the two representations cannot drift,
base_section_namerecovers the base section from it, andis_sensitive_optiontests a pair directly, then via the base section, then via the tail an
environment variable contributes -- so it can only ever recognise more options as
sensitive, never fewer.
The team name is not parsed out of an environment variable name, because a team
name may contain underscores; the name is matched against the tail each
registered option contributes instead. The config file section name is split on
the last separator, so the base section is recovered even for a team name that
contains the separator itself.
Four call sites use the predicate: the masking pass, the environment collection,
write, and the single option config route, which does not go throughas_dictand so needed its own change.
Team scoped
_cmdand_secretentries are hidden in place rather than resolvedinto their value, because resolving them is not supported for a team.
Generated-by: Claude Opus 5 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
Add newsfragment for the team scoped sensitive option masking change
Keep one copy of the team scoped masking rationale, and record the untouched gap
The per-key secrets-backend options are matched by literal section name and stay
unaware of a team scoped spelling. Nothing leaks while the secrets backend itself
is not team aware, so it is recorded rather than fixed here.
Co-authored-by: Amogh Desai amoghrajesh1999@gmail.com
Co-authored-by: Amogh Desai amoghrajesh1999@gmail.com
(cherry picked from commit d41ac7b)
Co-authored-by: Jarek Potiuk jarek@potiuk.com
Co-authored-by: Amogh Desai amoghrajesh1999@gmail.com