Repository navigation
risk formulas: audit language-tier membership (embedded_python lands tier3) + decide how rosetta bias reporting treats documented Fc/Irc tier constants #2653
Description
Found by the #2560 rosetta classification sweep while tracing why §3 risk metrics deviate for languages whose §2 raw signals are exactly on plant.
What happens
signal_processor.py's _get_tier (~L2029) assigns per-language scoring constants by exact string membership:
TIER_VARS = {"tier1": {"fc": 1.0, "irc": 0}, "tier2": {"fc": 0.85, "irc": 2}, "tier3": {"fc": 0.60, "irc": 5}}
explicit = {"rust","go","swift","java","typescript","csharp","dart"} # tier1
structured = {"python","javascript","cpp","c","ruby","kotlin","php"} # tier2
# everything else -> tier3irc is added as a flat risk term (irc/safe_loc in _calc_cog_load; raw irc in _calc_documentation's risk_hits) and fc scales down doc/ownership defense credit — so the ~32 tier3 languages get an automatic risk floor and ~30% less documentation credit than python for byte-identical content. Notably embedded_python is not the literal string "python", so it lands tier3 despite being python.
Why it matters
On the rosetta control corpus (identical planted intent in every language) this produces systematic §3 risk-band deviations for tier3 languages whose §2 raw signals are exactly on plant — i.e. measured cross-language bias attributable to the tier constants alone, not to anything in the scanned code. Related observation from the same sweep: risk_tech_debt swings −50%..+100% across six config/DSL languages whose raw debt plants are byte-for-byte identical (1 planned + 1 fragile), purely from LOC normalization — the two mechanisms compound.
Ask
Probably intentional in spirit (implicit-risk correction for loosely-typed/legacy languages), but worth (a) documenting the tier lists as a deliberate, reviewable table rather than inline sets, (b) auditing membership (embedded_python→tier2 at minimum; scala/groovy/haskell/zig arguably tier1/2-shaped), and (c) deciding whether the rosetta bias report should normalize tiers out when comparing languages, since it currently reads tier constants as cross-language bias.
Found via keyword-rosetta classification (cause-keys tier-constant-risk-floor, tech-debt-score-loc-normalization-artifact).
Activity
squid-protocol commentedon Sep 2, 2026
Correction/narrowing after reading the methodology wiki (docs/wiki/08-01 through 08-03): the tier mechanism itself is documented, deliberate design — 08-03's Universal Exposure Framework specifies Language Confidence Tiers applying Fidelity Coefficients (Fc) and Implicit Risk Corrections (Irc) "based on language strictness," and explicitly owns the tradeoff ("generalize thousands of languages into three buckets, reducing precision for niche languages"). So this is NOT an undocumented surprise, and the issue title's "unaudited" applies to the membership sets, not the mechanism.
What remains actionable, narrowed:
embedded_pythonlands tier3 because it isn't the literal string"python"— but it IS python by the 08-03 "language strictness" criterion; that looks like a plain oversight. Several others are worth a pass (scala/groovy/haskell/zig vs the tier1/tier2 sets).[-]risk formulas: hardcoded language-tier constants (fc/irc) create systematic cross-language risk bias; tier membership unaudited[/-][+]risk formulas: audit language-tier membership (embedded_python lands tier3) + decide how rosetta bias reporting treats documented Fc/Irc tier constants[/+]squid-protocol commentedon Sep 4, 2026
Design pass done and implemented on the corpus side (gitgalaxy#2669 F.3) — keyword-rosetta#50. Decision (a) from #2669: the bias report normalises the tier constants out; the engine is not changed by this comment. Membership audit below asks for one decision.
What the corpus measured
With every planted input on the median, the seven tier-1 languages read
risk_tech_debt= 29.878, byte-identical, against a global median of 46.57;risk_documentationsteps 29.90 (tier1) / 40.84 (tier2) / 55.81 (tier3) withdocandownershipboth on plant. That is_get_tier's constants, and nothing else: the formulas that read them are exactly the seven whose bodies referenceirc/fc/ot—cognitive_load,safety_score,tech_debt,documentation,verification,concurrency,state_flux(read offsignal_processor.pyby AST at regen time,_registry.risk_dependencies()[m]["tier"], so a refactor surfaces as a derivation change rather than a stale list).What "normalise out" means, concretely
There is no clean division: the tier enters additively (
irc / mass_loc,irc * irc_mult), multiplicatively (fcon defence credit,oton verification) and then through a sigmoid. Re-scoring every language at one tier would need either an engine config hook or a second scan per language. The honest engine-free form is stratified reference medians: a tier-reading metric is banded against the median of the language's own tier (7 / 7 / 32 languages — the same program seven times is still a median), every other metric keeps the global median, and the per-tier medians are printed in the report as the documented offset rather than hidden. The tier map is stored indocs/bias_data.jsonastiers, the metric list astier_sensitive.Effect, engine
c3ca9892: 33risk_*cells were unexplained with every planted input in band. Within-tier banding turns the 14 tier-shaped ones green (risk_tech_debt×7 tier1;risk_documentationcsharp, dart, swift, kotlin, php, ruby;risk_cognitive_loaddart, kotlin, php) and moves four in-band cells out of band (risk_safety_scorec/javascript/python,risk_state_fluxcpp), all four already explained as derived. Caching the unplanted risk inputs explains 10 more by naming the input (risk_state_fluxdart/kotlin/scala/swift ←immutability_locks;risk_concurrencyagc/swift ←concurrency/sync_locks; yacc ←reflection_metaprogramming). Corpus-wide unexplained 91 → 67. The nine risk cells left are the api-zero family (risk_api_exposure×5,risk_documentation×4: adjustedapi= 0 because orphan→api conversion never fired), which is F.4's orphan census, not a tier question.Held equal already in the length-leak check since keyword-rosetta#48 (where
cog_rawfell from −0.63 to −0.55 the moment tier was held — most of that "length leak" wasirc / mass_loc).Membership audit (the decision requested)
embedded_python_get_tiermatches the literal"python"onlyscalahaskellzigkotlinvsswiftRecommendation:
embedded_python→ tier2 now (it is a straightforward misclassification by string match), and leave the arguable rows to the wiki's owner — each moves every risk score for that language in the golden master and needs a bless, and "arguably tier 2" is not the bar for that. Blast radius forembedded_python:language-cruciblehas anembedded_pythongroup, so the golden master re-blesses for every file in it (risk columns only; no extraction signal moves). One-line engine change in_get_tier, its own PR, its own bless.What this does not do
It does not decide whether the tier mechanism is right. It records it as design, measures its size (the per-tier median table), and stops the corpus from reporting it as bias. If the mechanism is ever revisited, the corpus is the fixture that shows the offset directly.
squid-protocol commentedon Sep 4, 2026
Design pass approved 2026-09-04 → epic #2716. This issue is narrowed back to the membership audit it opened as, and is closed by #2718 (
embedded_pythonresolves to python's row via a family map; the tier lists and the deadFIDELITY_TIERSare replaced by a documented strictness table + a fidelity table generated from keyword-rosetta). The safety sign inversion found on the way is #2717.feat(core-engine): language-strictness table + measured fidelity coef…