Skip to content

[Config Phase 0] Decide precedence order and merge semantics before building the resolver #332

Closed
Closed
[Config Phase 0] Decide precedence order and merge semantics before building the resolver#332
Labels
core-engineModifications to the central physics and parsing engineepicLarge-scale architectural milestone or multi-issue initiativepriority: highCore feature broken, but workarounds exist
Milestone
Quality of Life Update 🛠️ — Config, CI & Dev Tooling

Description

@squid-protocol
squid-protocol
opened on Jul 23, 2026

Context

#247 flagged "YAML and CLI overrides align better." Investigation found the
divergence is worse than a simple two-way mismatch -- there are currently
four separate, inconsistent override mechanisms:

  1. gitgalaxy_config.py module constants (~18 of them: STRICT_IMPORT_MODE,
    APPROVED_IMPORTS, BLACKLISTED_IMPORTS, APERTURE_CONFIG, GUIDESTAR_CONFIG,
    etc.), imported by value at load time by 4 files (galaxyscope.py,
    guidestar_lens.py, language_lens.py, supply_chain_firewall.py) --
    permanently disconnected from anything that happens after import.
  2. A generic .galaxyscope.yaml -> args.* interceptor in galaxyscope.py's
    main(), which currently reaches nothing, because no argparse flag exists
    for any gitgalaxy_config.py constant.
  3. A separate, hand-coded YAML merge that ONLY touches APERTURE_CONFIG, and
    only 4 of its keys (IGNORED_DIRECTORIES, CONTRABAND_PATTERNS,
    VENDOR_MINIFICATION_PATHS, SECRETS_EXACT).
  4. PROJECT_OVERRIDES -- a static Python dict in gitgalaxy_config.py, keyed
    by project folder name, merged into the same two locals as (3).

Only two local variables in main() (merged_langs, merged_aperture) ever
receive any of this. The three files that import gitgalaxy_config.py
directly (guidestar_lens.py, language_lens.py, supply_chain_firewall.py)
never see any override, ever -- confirmed via import-site inspection.

This issue's deliverable

A written decision (as a markdown doc or a detailed issue comment, not code)
covering:

  1. Full key inventory. Every gitgalaxy_config.py constant, and for each:
    should it be user-overridable at all? Via YAML, CLI, both?
  2. Precedence order. Explicit ranking, e.g.
    CLI flag > per-project PROJECT_OVERRIDES-by-name > .galaxyscope.yaml >
    gitgalaxy_config.py default -- or whatever order is actually intended.
    Must resolve the current unfixable case: a boolean set True by YAML can
    never be forced back to False by a store_true CLI flag.
  3. Merge semantics per key, not just per-source. Lists/sets (e.g.
    BLACKLISTED_IMPORTS) -- does a YAML override extend the default, or
    replace it entirely? Both are legitimate depending on the key; needs to
    be explicit per key, not a single blanket rule.
  4. Unknown-key handling. A YAML key that doesn't match anything should
    fail loudly (error, not silent drop) -- decide the exact failure mode
    (hard error vs. warning-and-ignore).
  5. Scope of PROJECT_OVERRIDES. Does this stay a Python-only mechanism,
    or does it become YAML-editable too? If it stays Python-only, is that
    a deliberate choice (e.g. for maintainer-curated known-project dialects
    vs. arbitrary user config)?

Explicitly out of scope

No code changes in this issue. Phases 1-3 (separate issues) implement
whatever gets decided here.

Activity

  1. squid-protocol
    added
    epicLarge-scale architectural milestone or multi-issue initiative
    core-engineModifications to the central physics and parsing engine
    priority: highCore feature broken, but workarounds exist
    on Jul 23, 2026

  2. squid-protocol commented on Jul 23, 2026

    @squid-protocol
    squid-protocol
    on Jul 23, 2026
    OwnerAuthor

    Phase 0 Decision: Config precedence, merge semantics, and scope

    Audited against the actual code on fix/collision-resolved-confidence-clamp
    (commit 437007b), not just the symptoms described in the original issue.
    Several things below correct or extend the issue's premise — see
    "Corrections to the original writeup" first.

    Corrections to the original writeup

    1. PROJECT_OVERRIDES is not in gitgalaxy_config.py. It lives in
      gitgalaxy/standards/language_standards.py:10405. It is also not the
      same kind of thing as the rest of the config — it's maintainer-curated,
      per-project language-dialect patches (extension lists, compiled regex
      overrides, directory shields) keyed by folder name (freebsd-src,
      cpython, wrf-fortran, Apollo-11, AppFlowy, ansible, bugzilla,
      ...). Decision below: it stays a separate, Python-only mechanism, not
      a 4th tier of the general resolver.

    2. 5 files have the unreachable-direct-import bug, not 3. In addition to
      guidestar_lens.py, language_lens.py, supply_chain_firewall.py:

      • gitgalaxy/tools/supply_chain_security/vault_sentinel.py (imports
        ALLOWLIST_PATHS, DENYLIST_PATTERNS)
      • gitgalaxy/tools/supply_chain_security/binary_anomaly_detector.py
        (imports ALLOWLIST_PATHS, DENYLIST_PATTERNS, XRAY_BYPASS_EXTENSIONS,
        XRAY_BYPASS_PATHS)

      Both use the same try: from gitgalaxy.standards.gitgalaxy_config import (...) except ImportError: <hardcoded fallback> pattern. Phase 3's scope
      should expand to include these two files.

    3. Two more consumers were never on anyone's radar and are also broken:

      • chronometer.py:23 does from gitgalaxy.standards import gitgalaxy_config as config, then reads CHRONOMETER_CONFIG and
        APERTURE_CONFIG off that raw module object — not the merged
        full_config that galaxyscope.py's main() builds. So even the 4
        APERTURE_CONFIG keys that main() does correctly merge today never
        reach the Chronometer. Same unreachable-override bug, different shape.
      • gpu_recorder.py:23,364 does from gitgalaxy.standards import gitgalaxy_config and reads PROJECT_STORIES directly off it. Same bug.

      These need to be added to the Phase 3 file list (or a Phase 3b).

    4. STATIC_ARCHETYPES doesn't even reach its own default — this is a
      live bug, not an override gap.       signal_processor.py:19 does from gitgalaxy.standards import analysis_lens as config, then at lines
      338/382/428 reads getattr(config, "STATIC_ARCHETYPES", {}).
      STATIC_ARCHETYPES is defined in gitgalaxy_config.py, not
      analysis_lens.py — the getattr fallback of {} fires unconditionally.
      Archetype labeling via this dict has never worked, independent of any
      override mechanism. Recommend filing this as its own bug ticket
      (wrong module aliased) rather than folding the fix into the Phase 1/2/3
      resolver work — it's an "override never reaches" story, this is a
      "default never loads" story, and conflating them will make the resolver
      PR harder to review.

    5. ORCHESTRATOR_RULES / POPULARITY_STOP_STEMS is dead code. Grepped
      with no consumer anywhere outside its own definition. Recommend deleting
      it in Phase 1 rather than building override plumbing for a key nothing
      reads, or filing a one-line follow-up if someone confirms it's meant to
      be wired up somewhere.

    6. There's already an undocumented precedent for YAML-only keys with no
      gitgalaxy_config.py default at all      : SARIF_IGNORED_RULES and
      SARIF_IGNORED_PATHS are read straight out of config_file_data in
      main() (lines 2680-2681) with no CLI flag and no module constant. The
      repo's own .galaxyscope.yaml uses exactly these two keys and nothing
      else. The resolver should accommodate config keys that are YAML/CLI-only
      with no compiled-in default, not assume every key originates in
      gitgalaxy_config.py.

    Decisions

    Precedence order

    CLI flag
  overrides
.galaxyscope.yaml
  overrides
gitgalaxy_config.py default

    (PROJECT_OVERRIDES dialect patches are explicitly not in this chain —
    see "Scope of PROJECT_OVERRIDES" below. They apply as a separate,
    always-additive patch keyed by project name, orthogonal to CLI/YAML/default.)

    Fixing the store-true-can't-force-False footgun: the current code
    detects "was this overridden" by checking getattr(args, arg_key) in (None, False, 0.0, "") — i.e. it can't distinguish "user didn't pass --paranoid"
    from "user explicitly wants paranoid mode off." Phase 1 must replace
    action="store_true" with tri-state flags for anything that YAML can also
    set: argparse default None (not False), so None unambiguously means
    "not specified on the CLI" and only then does YAML get a chance to fill it
    in, and only when YAML doesn't set it either does the gitgalaxy_config.py
    default apply. This resolves the general precedence rule as "first source
    in the chain that actually specifies a value wins" rather than "first
    source with a truthy value wins."

    Unknown-key handling

    Hard error. An unrecognized key in .galaxyscope.yaml should raise and
    refuse to run the scan, not warn-and-continue. Justification: this is a
    security-policy config file. A typo'd STRICT_IMPORT_MODE (e.g.
    STRICT_IMPORT_MDOE) that silently no-ops leaves the supply-chain firewall
    in audit mode while the user believes they've turned on strict mode — a
    silent security regression is worse than a scan that refuses to start.
    This is also literally the failure mode #247 was filed to fix; a
    warn-and-ignore resolver would just relocate the same silent-drop bug one
    layer up instead of closing it.

    Scope of PROJECT_OVERRIDES

    Stays Python-only, and stays out of the general resolver's precedence
    chain entirely. It's maintainer-curated dialect knowledge for specific,
    known third-party repos gitgalaxy has been pointed at before — it includes
    compiled regex objects (e.g. wrf-fortran's concurrency pattern), which
    isn't something that belongs in a YAML file a user hand-edits. Treat it as
    a distinct "known project fixups" mechanism, applied unconditionally after
    CLI/YAML/default resolution, keyed by project_name — not a 4th precedence
    tier and not YAML-editable. If a future need arises for user-authored
    per-project overrides, that should be a new, separate mechanism rather than
    extending this one.

    Full key inventory

    Legend: Reach = does an override from any source actually get to the
    consumer today. Merge = recommended semantics once Phase 1 exists.

    Key Consumer(s) Today's reach Recommended override sources Merge semantics
    STRICT_IMPORT_MODE supply_chain_firewall.py (direct import — broken) Never YAML + CLI (--strict-imports, tri-state) Replace
    APPROVED_IMPORTS supply_chain_firewall.py (direct import — broken) Never YAML only Extend (union)
    BLACKLISTED_IMPORTS supply_chain_firewall.py (direct import — broken) Never YAML only Extend (union)
    FIREWALL_NETWORK_WEIGHTING supply_chain_firewall.py (direct import — broken) Never YAML + CLI (tri-state) Replace
    DENYLIST_PATTERNS binary_anomaly_detector.py, vault_sentinel.py (direct import — broken) Never YAML only Extend
    ALLOWLIST_PATHS binary_anomaly_detector.py, vault_sentinel.py, supply_chain_firewall.py (direct import — broken) Never YAML only Extend
    XRAY_BYPASS_EXTENSIONS binary_anomaly_detector.py (direct import — broken) Never YAML only Extend
    XRAY_BYPASS_PATHS binary_anomaly_detector.py (direct import — broken) Never YAML only Extend
    APERTURE_CONFIG.SECRETS_EXTENSIONS aperture.py (via full_config) Never (not in today's 4-key hand-merge) YAML only Extend
    APERTURE_CONFIG.SECRETS_EXACT aperture.py (via full_config) Yes (today's hand-merge) YAML only Extend
    APERTURE_CONFIG.IGNORED_DIRECTORIES aperture.py, chronometer.py (2nd path broken) Partial (reaches aperture.py only) YAML only Extend
    APERTURE_CONFIG.IGNORED_EXTENSIONS aperture.py (via full_config) Never (not in today's hand-merge) YAML only Extend
    APERTURE_CONFIG.CONTRABAND_PATTERNS aperture.py (via full_config) Yes (today's hand-merge) YAML only Extend
    APERTURE_CONFIG.VENDOR_MINIFICATION_PATHS aperture.py (via full_config) Yes (today's hand-merge) YAML only Extend
    APERTURE_CONFIG.MAX_LINE_LENGTH / MINIFICATION_SCAN_LIMIT / MAX_FILE_SIZE_MB aperture.py (via full_config) Never YAML + CLI (scalar) Replace
    APERTURE_CONFIG.BANDS aperture.py (via full_config) N/A Not user-overridable (internal label taxonomy) —
    PRIORITY_WHITELIST guidestar_lens.py (via constructor, but no merge logic exists) Passed through unmerged YAML only Extend
    GUIDESTAR_CONFIG.MANIFEST_MAP / INTENT_BIASED_SECTORS / EXEC_PREFIX_MAP guidestar_lens.py (direct import — broken) Never YAML only Extend (dict.update / union)
    LEXICAL_FAMILY_HEURISTICS language_lens.py (via constructor, no merge logic) Passed through unmerged Not user-facing (internal heuristic table) —
    EXACT_FILE_MATCH language_lens.py (direct import — broken) Never YAML only Extend (dict.update)
    TEST_NAMING_CONVENTIONS galaxyscope.py (via full_config, no merge logic) Passed through unmerged YAML only Extend
    CHRONOMETER_CONFIG chronometer.py (direct module import — broken) Never YAML only Extend (dict.update)
    STATIC_ARCHETYPES signal_processor.py (wrong module aliased — dead, see correction #4) Never (bug, not override gap) Fix the bug first; then YAML-extendable Extend
    PROJECT_STORIES gpu_recorder.py (direct import — broken) Never YAML only Extend (dict.update)
    ORCHESTRATOR_RULES / POPULARITY_STOP_STEMS (no consumer found) N/A — dead code Delete, or file follow-up if intentional —
    SARIF_IGNORED_RULES / SARIF_IGNORED_PATHS galaxyscope.py main() (YAML-only today, no default) Yes — already works YAML only (no gitgalaxy_config.py default; keep as-is) Extend
    PROJECT_OVERRIDES (in language_standards.py) galaxyscope.py main() Yes (project-name keyed) Not part of this resolver — Python-only, see decision above Extend (unconditional, after CLI/YAML/default resolve)

    What Phase 1 needs to do differently from what the original issue assumed

    • The resolver can't just generalize the 4-key APERTURE_CONFIG hand-merge
      pattern to "every key in gitgalaxy_config.py" — it also has to become
      the thing that chronometer.py, gpu_recorder.py, guidestar_lens.py,
      language_lens.py, supply_chain_firewall.py, vault_sentinel.py, and
      binary_anomaly_detector.py all pull from, replacing 7 independent
      direct-import sites, not the 3 the issue named. That's really a Phase 1 +
      Phase 3 scope correction — recommend updating issue [Config Phase 3] Migrate guidestar_lens.py, language_lens.py, supply_chain_firewall.py off direct gitgalaxy_config.py imports #335's file list
      before starting that phase.
    • STATIC_ARCHETYPES and ORCHESTRATOR_RULES should be pulled out of
      Phase 1's key-by-key merge work — one's a pre-existing bug unrelated to
      overrides, the other looks like dead code. Fixing/deleting them inside
      the resolver PR will make it harder to review; handle separately.

    Explicitly out of scope (unchanged from original)

    No code in this issue. Phases 1-3 implement the above.

  3. squid-protocol
    added 4 commits that reference this issue on Jul 23, 2026

    [Config Phase 1] Add config_resolver.py: single ResolvedConfig merge …

    19c2312

    Migrate galaxyscope.py main() to use the ResolvedConfig resolver (#334)

    3ebf13c

    Migrate galaxyscope.py main() to use the ResolvedConfig resolver (#334)…

    d6cc86f

    Migrate 7 direct gitgalaxy_config.py importers off module-level const…

    822673d
  4. squid-protocol
    closed this as completedon Jul 23, 2026
  5. squid-protocol
    added a commit that references this issue on Jul 23, 2026

    Fix CLI/YAML sentinel-collision in the pipeline-flag interceptor (#247)…

    dfea159
  6. squid-protocol
    added this to the Quality of Life Update 🛠️ — Config, CI & Dev Tooling milestone on Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    core-engineModifications to the central physics and parsing engineepicLarge-scale architectural milestone or multi-issue initiativepriority: highCore feature broken, but workarounds exist

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions