Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions ANTIGRAVITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,29 @@ GitGalaxy scans itself and outputs intelligence to `/docs/gitgalaxy_architecture
- Fastest path in an active session: `python tests/tools/self_scan.py` regenerates it in place.
- If you'd rather not run a local scan, the `gitgalaxy.yml` workflow's "Full Report" job now publishes a fresh copy as a `gitgalaxy-self-scan-db` build artifact on every merge to main -- pull the latest one from that workflow's most recent run instead.

## 7. Submitting Pull Requests
## 7. Extraction Hardening & Adversarial Testing

When tasked with "hardening extraction coverage" or similar testing epics, you must avoid **Self-Consistency Bias** (writing tests that merely pass against the *current implementation* instead of the *actual ground truth*).

To ensure rigorous, adversarial testing:
1. **Define "Valid" from Ground Truth First**: Before looking at the current regex implementation, determine what syntax is valid by consulting official language documentation, a ground-truth corpus, or other established rules in the same file (e.g. cross-referencing `branch`, `safety`, and `state_mutation` rules to find missing opcodes).
2. **Do Not Cement Implementation Flaws**: If a structurally valid syntax does not match the current regex, the regex is buggy, NOT the syntax. Never write a negative/invalid test to "prove" the regex correctly ignores it.
3. **Cross-Rule Validation**: Actively check sibling rules in `language_standards.py`. If `args` extracts registers, make sure it covers the instructions listed in `state_mutation` that mutate those registers.
4. **Corpus Grepping**: When in doubt about whether a syntax exists or how common it is, `grep` the actual corpus (e.g. in `language-crucible`) to find empirical evidence of the shape and frequency of certain keywords or patterns.
5. **Multi-Round Adversarial Checks**:
- *Round 1 (Breadth)*: Gather keywords, opcodes, and structures from external docs and sibling rules.
- *Round 2 (Regex)*: Check the regex against this gathered ground truth. Identify missing captures and false positives.
- *Round 3 (Corpus)*: Verify fixes against real corpus data using `crucible_check.py` to ensure the modifications match reality.

## 8. Submitting Pull Requests

When working in this repository, **always work on a side branch and submit a PR to `main`. Do not merge or push directly to `main`.**

Before submitting a Pull Request, you must run the following deterministic tools in your belt to test out and fix your changes, acting as your local CI pipeline:
1. **Unit Tests:** `venv/bin/python -m pytest tests/`
2. **Lint & Types (Ruff, Mypy):** `PATH="$PWD/venv/bin:$PATH" python tests/tools/audit_check.py` (run with `--regenerate` or run `ruff format .` if auto-fixes are needed).
3. **Golden Fixtures (Crucible):** `venv/bin/python tests/tools/crucible_check.py` (run with `--update --yes` if the output intentionally changed due to better parsing rules).

When generating or submitting a Pull Request for this repository, it is critical to provide comprehensive context for reviewers.
- **Always add a thorough description:** Outline the changes, the rationale, and any structural boundaries or tests added. Do not leave the PR body blank or sparse.
- **Always add a thorough description:** Outline the changes, the rationale, and any structural boundaries or tests added. Explain *why* any golden masters changed. Do not leave the PR body blank or sparse.
- **Add relevant labels:** Ensure the PR has descriptive labels attached so it integrates correctly into the project's tracking and CI processes.
10 changes: 5 additions & 5 deletions gitgalaxy/core/guidestar_lens.py
Original file line number Diff line number Diff line change
Expand Up @@ -274,9 +274,9 @@ def _parse_package_json(self, path: Path):
scripts = data.get("scripts", {})
for name, cmd in scripts.items():
files = re.findall(r"([a-zA-Z0-9_\-\./]+\.(?:js|ts|mjs|cjs))", cmd)
for f in files:
for file_path in files:
self._inject_intent_lock(
f, "javascript", 0.85, f"Manifest Script (package.json:scripts:{name})"
file_path, "javascript", 0.85, f"Manifest Script (package.json:scripts:{name})"
)
except Exception as e:
self.logger.debug(f"GuideStar: failed to parse manifest '{path}': {e}")
Expand All @@ -291,9 +291,9 @@ def _parse_makefile(self, path: Path):
matches = re.findall(r"(?:SRCS|SOURCES|FILES|TARGET)\s*[+:]?=\s*(.*)", content, re.I)
for m in matches:
files = m.split()
for f in files:
if "." in f:
self._inject_intent_lock(f, "unknown", 0.85, "Manifest Source (Makefile)")
for file_path in files:
if "." in file_path:
self._inject_intent_lock(file_path, "unknown", 0.85, "Manifest Source (Makefile)")

# Strategy 2: Find target lines like 'build: main.o'
targets = re.findall(r"^([a-zA-Z0-9_\-]+)\s*:", content, re.M)
Expand Down
13 changes: 8 additions & 5 deletions gitgalaxy/standards/language_standards.py
Original file line number Diff line number Diff line change
Expand Up @@ -6439,7 +6439,7 @@ class PrismConfigSchema(TypedDict):
# 2. args (Parameters / Coupling)
# Assembly uses ABI registers for parameter coupling.
"args": re.compile(
r"\b([er]di|[er]si|[er]dx|[er]cx|[er][89]|x[0-7]|w[0-7]|v[0-7]|xmm[0-7])\b",
r"\b([er]di|[er]si|[er]dx|[er]cx|[er][89]|x[0-7]|w[0-7]|v[0-7]|xmm[0-7]|r[0-7])\b",
re.I,
),
# 3. linear (Sequential Boundaries)
Expand All @@ -6451,12 +6451,15 @@ class PrismConfigSchema(TypedDict):
# 4. func_start (Executable Logic Anchors)
# Subroutine entry points. EXCLUDES data labels or local loop markers.
"func_start": re.compile(
r"^[ \t]*(?!\.L|\.LC|\d|\.text|\.data|\.bss)([a-zA-Z_][a-zA-Z0-9_.$]*)(?=[ \t]*:)",
r"^[ \t]*(?!\.L|\.LC|\d|\.text|\.data|\.bss)([a-zA-Z_?@.][a-zA-Z0-9_.$?@]*)(?=[ \t]*:)",
re.M,
),
# 5. class_start (Object / Entity Declarations)
# Maps to assembler structure definition macros.
"class_start": re.compile(r"^[ \t]*(?:struc|STRUCT|\.struct)\s+[a-zA-Z_]\w*", re.M | re.I),
"class_start": re.compile(
r"^[ \t]*(?:(?:struc|STRUCT|\.struct)\s+[a-zA-Z_?@.][a-zA-Z0-9_.$?@]*|[a-zA-Z_?@.][a-zA-Z0-9_.$?@]*\s+(?:struc|STRUCT|\.struct))\b",
re.M | re.I,
),
# --- PHASE 2: RISK & STRUCTURAL INTEGRITY ---
# 6. safety (Defensive Programming / Validation)
# Stack preservation and defensive frame setups.
Expand Down Expand Up @@ -6529,9 +6532,9 @@ class PrismConfigSchema(TypedDict):
# Complex SIB addressing and block replication.
"reflection_metaprogramming": re.compile(r"\[\s*[a-zA-Z0-9_]+\s*\+\s*[a-zA-Z0-9_]+\s*\*\s*\d+", re.I),
# 24. import (Dependency Inclusions)
"import": re.compile(r"^[ \t]*(?:%include|\.include|\.incbin)\b", re.M | re.I),
"import": re.compile(r"^[ \t]*(?:%include|\.include|\.incbin|INCLUDE|INCLUDELIB)\b", re.M | re.I),
"_dependency_capture": re.compile(
r"^[ \t]*(?:%include|\.include|\.incbin)\s+(?:['\"]([^'\"]+)['\"]|([^'\"\s]+))",
r"^[ \t]*(?:%include|\.include|\.incbin|INCLUDE|INCLUDELIB)\s+(?:['\"]([^'\"]+)['\"]|([^'\"\s]+))",
re.M | re.I,
),
# 25. ownership (Authorship Metadata)
Expand Down
21 changes: 21 additions & 0 deletions scratch.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import json
import subprocess


def get_json(rev):
out = subprocess.check_output(["git", "show", f"{rev}:tests/golden_master_zero_dep_audit.json"])
return json.loads(out)


old_data = get_json("HEAD~1")
new_data = get_json("HEAD")

files = new_data["6. Parsed Files (Scanned Artifacts)"]["assembly/bootos"]["Files"]
for path, new_f in files.items():
old_f = old_data["6. Parsed Files (Scanned Artifacts)"]["assembly/bootos"]["Files"][path]
print(f"\n{path} changes:")
old_sigs = old_f.get("7. Structural Signatures (Net Mitigated Signals)", {})
new_sigs = new_f.get("7. Structural Signatures (Net Mitigated Signals)", {})
for k, v in new_sigs.items():
if old_sigs.get(k) != v:
print(f" {k}: {old_sigs.get(k)} -> {v}")
4 changes: 1 addition & 3 deletions tests/core_engine/test_prism.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,9 +237,7 @@ def test_prism_format_and_xml_bypass(prism_engine):
def test_prism_php_string_extraction(prism_engine):
"""Proves PHP Heredoc strings are stripped to the documentation stream."""
prism_engine.languages["php"] = {"lexical_family": "standard_block"}
prism_engine.PHP_HEREDOC_PATTERN = re.compile(
r"<<<[\'\"]?([a-zA-Z0-9_]+)[\'\"]?\n(?:.*\n)*?\s*\1;?", re.M
)
prism_engine.PHP_HEREDOC_PATTERN = re.compile(r"<<<[\'\"]?([a-zA-Z0-9_]+)[\'\"]?\n(?:.*\n)*?\s*\1;?", re.M)

content = "<?php\n$a = <<<EOT\nMassive Text\nEOT;\n// comment"
res = prism_engine.split_streams(content, primary_lang="php")
Expand Down
6 changes: 6 additions & 0 deletions tests/extraction/how_to_harden_extraction.md
Original file line number Diff line number Diff line change
Expand Up @@ -541,6 +541,12 @@ specifically. Check every language against these *first* before assuming a rule
sibling, non-gauntlet-scoped rule with nearly the same pattern) was deliberately left unfixed --
out of the four-gauntlet scope for a given issue, not an oversight; don't feel obligated to fix
every structurally-similar sibling rule outside the four gauntlets in the same pass.
44. **(#856) A negative lookahead exclusion can be entirely neutralized by the character class that
immediately follows it.** Assembly's `func_start` tried to exclude local labels like `.L` via
`(?!\.L)([a-zA-Z_]...)`, but `[a-zA-Z_]` already rejected any identifier starting with a dot.
This meant legitimate global labels starting with `.` were blocked, and the lookahead was dead
code. When adding a negative lookahead, verify that the subsequent matching logic actually
permits the excluded shape to begin with.

## Process: the epic and its sub-issues

Expand Down
Loading
Loading