Skip to content

fix(#4106): compute file-archetype distance + fingerprint; retire the dead per-language fields - #4117

Merged
squid-protocol merged 2 commits into
mainfrom
fix/4106-file-archetype-fields
Oct 2, 2026
Merged

squid-protocol merged 2 commits into
mainfrom
fix/4106-file-archetype-fields

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Closes #4106.

Problem

Five per-file archetype fields were placeholders reported as measurements: constant N/A / 0.0 / {} on all 3,003 crucible code files. #3061 moved file classification into record_keeper, which only overwrites the label. Nothing has computed the distance or fingerprint since, and nothing applies the per-language model at all.

Fix

Computed: the general file archetype (k=18).

  • record_keeper._classify_file_archetype_detail returns the nearest archetype, its Euclidean distance to that centroid, and the distance to every centroid. The nearest-centroid loop already computed all of them and threw them away. They're now written to global_drift / archetype_fingerprint, which revives:
    • the GPU recorder's secondary-archetype a_ids / a_dists (the site reads only a_ids[i][0], so a secondary id is safe);
    • the LLM brief's "Top Global Matches" and its mixed-responsibility section.
  • Audit key Repository Drift (Z-Score) → Repository Drift (Centroid Distance). It's a distance, never a z-score. Nothing outside the recorder read the old key.
  • Mixed-responsibility is recalibrated. The old absolute delta <= 0.9 was tuned for a feature space retired in fix(archetypes): classify file archetype from a self-describing brain, post-assembly #3061: on today's fingerprints (median distance 6.1) it would flag 33% of crucible code files. It's now relative, ENGINE_CONSTANTS["MIXED_RESPONSIBILITY_GAP"] = 0.02 (second-nearest within 2% of the nearest), which flags ~5% (149 / 3,003). Both the LLM brief and the GPU recorder use the same constant.

Retired: the per-language "local micro-species" model.

  • SPECIFIC_FILE_INFERENCE_MODEL (~3,600 lines) and LANGUAGE_INFERENCE_MODELS are removed. Nothing applied them since fix(archetypes): classify file archetype from a self-describing brain, post-assembly #3061, and they can't be applied: there's no FEATURE_NAMES, and 74 dims match no feature space the engine computes (83 at fix(archetypes): classify file archetype from a self-describing brain, post-assembly #3061, 84 today). Reviving them means retraining as a self-describing brain.
  • local_archetype / local_drift / local_fingerprint are removed from:
    • telemetry;
    • the audit (File Archetype, File Drift (Z-Score), File Fingerprint);
    • the LLM brief, including the "Severe Anti-Patterns" drift-ratio check, which could never fire;
    • the LLM graph DB columns.
  • docs/wiki/08-19 no longer documents the drift-ratio multiplier. The page also flags that _calc_obscured_payload itself is no longer in signal_processor.py and needs re-verification.

Golden master

Re-blessed against crucible v1.6.0. Only 3_architectural_profile parts move in each fixture:

  • the four retired keys are deleted;
  • repository_fingerprint is now populated;
  • repository_drift_centroid_distance is added.

No label, signal, risk or summary value changed.

Tests

  • test_signal_processor: the detail call returns the correct distance and full fingerprint, and the nearest label matches the minimum distance.
  • test_audit_recorder: the distance is reported under its new key, and the retired keys are absent.
  • core_engine + tools_recorders: 2,407 passed; audit_check all clear. Every changed file compiles on Python 3.9.

Merge order

This touches llm_recorder.py / audit_recorder.py / record_keeper.py, as does #4113. Whichever merges second needs a merge from main (import-line and adjacent-hunk conflicts only).

🤖 Generated with Claude Code

…re the dead per-language fields

Five per-file archetype fields were placeholders reported as measurements --
constant N/A / 0.0 / {} on every crucible file since #3061 moved file
classification into record_keeper, which only overwrote the label.

Computed (general file archetype, k=18):
- record_keeper._classify_file_archetype_detail returns the nearest archetype,
  its Euclidean centroid distance and the distance to every centroid; the loop
  already computed them and threw them away. Written to global_drift and
  archetype_fingerprint, which revives the GPU a_ids/a_dists secondary match,
  the LLM brief's "Top Global Matches" and its mixed-responsibility section.
- The audit key "Repository Drift (Z-Score)" becomes
  "Repository Drift (Centroid Distance)" -- it is a distance, not a z-score.
- Mixed-responsibility now uses a RELATIVE gap (ENGINE_CONSTANTS
  MIXED_RESPONSIBILITY_GAP = 0.02: second-nearest within 2% of the nearest).
  The old absolute 0.9 was tuned for a retired feature space and would flag
  33% of crucible code files; 0.02 flags ~5% (149/3,003).

Retired (per-language "local micro-species"):
- SPECIFIC_FILE_INFERENCE_MODEL (~3,600 lines) and LANGUAGE_INFERENCE_MODELS
  are removed: nothing applied them since #3061, and they could not be applied
  -- no FEATURE_NAMES, and 74 dims match no feature space the engine computes.
- local_archetype/local_drift/local_fingerprint leave the telemetry, the audit
  ("File Archetype", "File Drift (Z-Score)", "File Fingerprint"), the LLM brief
  (incl. the "Severe Anti-Patterns" drift-ratio check) and the LLM graph DB.
- docs/wiki/08-19 no longer documents the drift-ratio multiplier.

Golden masters re-blessed: only 3_architectural_profile parts move -- four
retired keys deleted, repository_fingerprint populated, and the renamed
distance key added.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@squid-protocol squid-protocol added bug Unintended behavior or logic failure in the engine data-integrity Issues relating to RAM state or SQLite database fidelity core-engine Modifications to the central physics and parsing engine labels Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

…pe-fields

# Conflicts:
#	tests/tools_recorders/test_audit_recorder.py
@squid-protocol
squid-protocol merged commit 13a24d5 into main Oct 2, 2026
33 checks passed
@squid-protocol
squid-protocol deleted the fix/4106-file-archetype-fields branch October 2, 2026 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Unintended behavior or logic failure in the engine core-engine Modifications to the central physics and parsing engine data-integrity Issues relating to RAM state or SQLite database fidelity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dead per-file archetype fields: File Archetype / drift z / fingerprints are never computed but reported as measured

1 participant