Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 231 additions & 0 deletions docs/args_rule_contract.md

Large diffs are not rendered by default.

49 changes: 49 additions & 0 deletions docs/why_gitgalaxy_beats_ast_here.md
Original file line number Diff line number Diff line change
Expand Up @@ -996,6 +996,55 @@ clause-local count.
matlab, scala, swift) — each of those has its own validated ledger verdict (all "GitGalaxy
counts args accurately"), none of them this eta-reduction mechanism.

## Claim 15: reading a method declaration split by a preprocessor conditional (Objective-C)

Found 2026-09-05 while auditing the `args` rule for
[#2773](https://github.com/squid-protocol/gitgalaxy/issues/2773), using a `tree-sitter` `objc`
parse of the engine's own code stream as the oracle for "is this hit a declaration or a call?".

`language-crucible/data/objective-c/worldwideweb/HyperText.m:1426` is a real method definition
whose body is guarded by a preprocessor conditional placed *between* the selector and the opening
brace — idiomatic NeXTSTEP-era style for keeping a superseded implementation alongside its
replacement:

```objc
- keyDown:(NXEvent*)theEvent
#ifdef TRY1
{
id result;
...
```

`tree-sitter-objc` cannot parse it. Parsing that file's code stream produces an `ERROR` node
starting exactly at `- keyDown:`, and **no `method_definition`/`method_declaration` node covers
line 1426** — the method is simply absent from the tree (48 method nodes for ~49 line-leading
`-`/`+` heads; the file's other two `ERROR` regions do still resolve to method nodes). GitGalaxy
reads it correctly: `func_start` matches the method, and after #2773 `args` records its one
parameter, because both anchor on the `-`/`+` selector head and neither requires the body brace to
be lexically adjacent.

The mechanism is the same one Claim 3 (`csharp` `ref struct`) and Claim 13 describe — a grammar
that requires a shape stricter than the real dialect, then loses a whole region to error recovery
— applied to a construct the grammar has no concept of at all. It is worth recording separately
because a preprocessor directive can appear at *any* point in a C-family declaration, so the same
failure is reachable in `c`, `cpp` and `objective-c` without the file being unusual in any other
way.

## Where Claim 15 does NOT apply

- It is not a claim that GitGalaxy parses Objective-C better than tree-sitter in general. On the
same file, tree-sitter's parse was the oracle that found 277 `args` false positives in
GitGalaxy's own rule, which is what #2773 fixed. The claim is about this construct only.
- It is not a claim about `#if`-guarded code in general — GitGalaxy has no preprocessor evaluation
either, and a declaration that only exists in one branch of an `#if`/`#else` pair is counted in
both. This is specifically about a directive interrupting a declaration that exists
unconditionally.
- It was found on one corpus file, and it is one method. The surrounding count is that file's own,
not a measured cross-corpus recall gap. `tree_sitter_accuracy_audit.py` does baseline
`objective-c`, but only over the functions both readers agree exist — a function tree-sitter
never emits cannot appear in `args_comparable` — so there is no live metric backing this the way
Claim 8 is backed by `ts_extra_functions`.

## Where this doc is used

- README.md's "One Graph, Not Five Separate Tools" section links here as the narrow exceptions to
Expand Down
2 changes: 1 addition & 1 deletion gitgalaxy/standards/how_to_add_a_language.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ Generate a valid Python dictionary matching this exact structure.
# --- PHASE 1: LOGIC TOPOLOGY & STRUCTURE ---
# branch: Control flow that forces the CPU to make a decision or jump. Includes: if, else, switch, for, while, catch, try, &&, ||, ternary. EXCLUDES: Exceptions (throw, raise) — these belong in panics_and_aborts.
"branch": re.compile(r""),
# args: Signatures defining input parameters. Includes: parameter blocks of functions, methods, and lambdas. Must safely step over type hints.
# args: THE PARAMETERS A CALLABLE DECLARES. Includes: the parameter blocks of functions, methods, constructors and lambdas; must safely step over type hints. Anchor the parameter list to the declaration it belongs to -- a declaration keyword (`def`/`fn`/`proc`), a mandatory return type, a terminator lookahead on what FOLLOWS the list (`(?=\{)`, `(?=:|\{)`), or a typed parameter list -- because `(...)` is equally a call, a cast and a grouped expression. EXCLUDES call sites: a call consumes a parameter surface, it does not declare one, and unlike `func_start` there is no `_slice_by_braces` downstream to drop the ones without a body. Where the language has no formal parameter list, match the construct that stands in for a declared parameter and record it in the fallback table. Full contract, corollaries and the 46-language audit: docs/args_rule_contract.md (#2773).
"args": re.compile(r""),
# structural_boundaries: Keywords defining structural boundaries and straight-line execution. Includes: var, return, class, import. EXCLUDES: Access modifiers (public, private) and Immutability keywords (const, final — these belong in immutability_locks).
"structural_boundaries": re.compile(r""),
Expand Down
59 changes: 58 additions & 1 deletion gitgalaxy/standards/language_standards/languages/objectivec.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,64 @@
# documented, pipeline-shielded limitation, the same shape as
# the pre-existing comment/string-lookalike one just below in
# this same file's test suite.
r"((?:(?:[a-zA-Z_]\w{0,80}[ \t\n]*)?:\s*\([^()]*(?:\([^()]*(?:\([^()]*\)[^()]*)*\)[^()]*)*\)\s*[a-zA-Z_]\w*[ \t\n]*|[a-zA-Z_]\w{0,80}[ \t\n]*:\s*[a-zA-Z_]\w*[ \t\n]*)+)|\^[ \t]*([a-zA-Z_]\w*\s*)?(\([^()]*(?:\([^()]*(?:\([^()]*\)[^()]*)*\)[^()]*)*\))|(?!(?:if|for|while|switch|catch|return|sizeof)\b)\b([a-zA-Z_]\w*)[ \t\n]*(\([^()]*(?:\([^()]*(?:\([^()]*\)[^()]*)*\)[^()]*)*\))[ \t\n]*(?:\{|;)",
#
# #2773: both arms were counting CALL SITES, not declared
# parameters -- `args` counts the parameters a callable
# DECLARES (docs/args_rule_contract.md), and a call consumes a
# parameter surface rather than publishing one. Measured over
# the code stream of language-crucible/data/objective-c
# (worldwideweb), classified against a tree-sitter `objc`
# parse of the same stream: 378 hits, only 101 of them real
# declarations (26.7% precision).
# * The colon-selector arm matched every Objective-C MESSAGE
# SEND -- `[store setVersion:ANCHOR_CURRENT_VERSION]`,
# `[list objectAt:i]` -- because a send is lexically
# identical to the untyped `label:name` parameter shape
# (#1335). 120 of the 277 false hits, plus 3 ternaries
# (`cond ? MIN_HEIGHT : maxY`) and both `@interface
# Anchor:Object` superclass colons. Fixed by requiring the
# span to lead with a real method-declaration head,
# `^[-+]` plus an optional return type -- the one context
# a keyword-message SIGNATURE can appear in and a send
# never can (a send always leads with `[`). This also
# closes the string-literal hole the issue asked for a
# negative test on: prism strips comments, but not
# strings, so `@"status: ok"` used to be an `args` hit.
# * The plain C arm accepted ANY `name(...)` followed by
# `{`/`;`, so every bare call statement -- `free(conn);`,
# `abort();`, `printf("...", x);` -- scored an argument.
# 146 of the false hits. Fixed by demanding the same
# structural proof c/cpp's own args rules demand: the
# parameter list must OPEN with a type token, and (unlike
# c/cpp, which stop at the type) that type must be
# followed by a parameter NAME or a `*`/`&` -- without the
# name requirement `StrAllocCopy(Address, tag);` still
# read as typed, since `Address` satisfies the PascalCase
# typedef fallback all three rules share.
# After both: 378 -> 101 hits, 100 of them real declarations
# (the 101st is a method definition tree-sitter itself
# mis-parses). The only real declaration dropped is
# `page_width()` -- an empty parameter list, i.e. no parameter
# surface to count, exactly as c/cpp already treat it.
# arm 1 -- a keyword-message method SIGNATURE: the `-`/`+` lead (with
# func_start's own optional macro/`__attribute__` prefix and an optional
# `(return type)`) is what separates a declaration from a message send.
r"^[ \t]*(?:[A-Z_0-9]+[ \t]+|__attribute__[ \t]*\([^()]*(?:\([^()]*(?:\([^()]*\)[^()]*)*\)[^()]*)*\)[ \t]+)*"
r"[-+][ \t\n]*(?:\([^()]*(?:\([^()]*(?:\([^()]*\)[^()]*)*\)[^()]*)*\)[ \t\n]*)?"
r"((?:(?:[a-zA-Z_]\w{0,80}[ \t\n]*)?:\s*\([^()]*(?:\([^()]*(?:\([^()]*\)[^()]*)*\)[^()]*)*\)\s*[a-zA-Z_]\w*[ \t\n]*"
r"|[a-zA-Z_]\w{0,80}[ \t\n]*:\s*[a-zA-Z_]\w*[ \t\n]*)+)|"
# arm 2 -- a block literal's own parameter list, `^(int x){ ... }`.
r"\^[ \t]*([a-zA-Z_]\w*\s*)?(\([^()]*(?:\([^()]*(?:\([^()]*\)[^()]*)*\)[^()]*)*\))|"
# arm 3 -- a plain C function declaration/definition. The parameter list
# must open with a type token AND that token must be followed by a
# parameter name or a `*`/`&`, which is what makes it a declaration
# rather than a call whose first argument happens to be capitalised.
r"(?!(?:if|for|while|switch|catch|return|sizeof)\b)\b([a-zA-Z_]\w*)[ \t\n]*"
r"(\(\s*(?:(?:const|volatile|__strong|__weak|__unsafe_unretained|_Nullable|_Nonnull)\s+)*"
r"(?:void\s*\)|(?:int|char|void|float|double|long|short|unsigned|signed|struct|enum|union"
r"|id|BOOL|SEL|IMP|Class|instancetype|_*[A-Z]\w*|[a-z_]\w*_t|[a-z_]\w*)\b"
r"[ \t\n]*[*&]*[ \t\n]*[a-zA-Z_]\w*(?:[^)(]|\([^()]*(?:\([^()]*(?:\([^()]*\)[^()]*)*\)[^()]*)*\))*\)))"
r"[ \t\n]*(?:\{|;)",
re.M,
),
# Which `args` capture-group index represents an objc
Expand Down
41 changes: 39 additions & 2 deletions gitgalaxy/standards/language_standards/languages/typescript.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,52 @@
# 1/4 too, purely so existing extraction tests keep passing.
"args": re.compile(
r"function\s+(\w*)(?:[ \t\n]{0,50}<(?:[^<>]|<[^<>]*>)*>)?[ \t\n]{0,50}(\((?:[^()]|\((?:[^()]|\([^()]*\))*\))*\))|"
r"(\((?:[^()]|\((?:[^()]|\([^()]*\))*\))*\))[^=;{]*=>|"
# #2773: the gap between an arrow's parameter list and its `=>` used to be
# `[^=;{]*` -- unbounded and newline-crossing, so ANY parenthesised
# expression matched as long as some `=>` turned up later on the way to the
# next `=`/`;`/`{`. `some((type as UnionType).types, t => ...)` scored the
# cast's own parens as a parameter list (384 such hits on the typescript
# crucible corpus). The only thing that can legally sit between `)` and `=>`
# is a return-type annotation, so the gap is now whitespace or a `:`-led
# annotation, both bounded. Bounding it also RECOVERS ~580 real arrow
# parameter lists this arm used to swallow inside an over-long match.
# The annotation itself may carry ONE level of balanced parens, because a
# curried arrow's return type is itself a function type -- fp-ts's
# `tryCatchK = <A, B, E>(f: ..., onThrow: ...): ((...a: A) => Either<E, B>) =>`
# (Either.ts:1406). Excluding `(` outright made this arm skip the real
# 2-parameter list and match the `(...a: A)` inside the annotation instead
# (6 functions, tree_sitter_accuracy_audit args_exact_match 2881 -> 2875).
# Newlines stay excluded on both sides so the gap cannot run down the file.
r"(\((?:[^()]|\((?:[^()]|\([^()]*\))*\))*\))[ \t\n]{0,50}(?::(?:[^=;{()\n]|\((?:[^()\n]|\([^()\n]*\))*\)){0,120})?=>|"
r"([a-zA-Z_$][\w$]{0,100})[ \t]*=>|"
# #2539: `with` excluded CONDITIONALLY (only when followed by whitespace
# then `(` -- the `with (shape) {` statement shape) rather than added to
# the unconditional keyword list: `with` is a real, prominent method name
# in modern code (`Array.prototype.with` ES2023, Temporal's `.with()`),
# and formatters never put a space before a method definition's paren --
# same discriminator func_start's #2276 catch/return/throw fix uses.
r"^[ \t]*(?:(?:public|private|protected|static|override|abstract|readonly)[ \t]+){0,4}(?:async[ \t]+)?(?:\*[ \t]*)?(?:get\s+|set[ \t]+)?(?!(?:if|for|while|switch|catch|return|throw|new|typeof|yield|await|void)\b|with\b[ \t\n]+\()(\[[^\]]+\]|[#]?[a-zA-Z_$][\w$]*)(?:[ \t\n]{0,50}<(?:[^<>]|<[^<>]*>)*>)?[ \t\n]{0,50}(\((?:[^()]|\((?:[^()]|\([^()]*\))*\))*\))",
# #2773: this arm had no declaration anchor at all -- `^[ \t]*IDENT(...)` is
# equally the shape of a bare CALL STATEMENT, and a call consumes a
# parameter surface rather than declaring one (docs/args_rule_contract.md).
# Measured over the code stream of language-crucible/data/typescript and
# classified against a tree-sitter parse of the same stream, this arm
# produced 6133 call sites against 3026 real declarations -- 67% wrong, and
# `describe(kit);`/`expect(kit);` in the keyword-rosetta shell were two of
# them. What follows the parameter list separates the two cleanly: a
# declaration is followed by its body `{` or by a `:` return-type
# annotation (2900 of the 3026), a call statement by `;`, `,`, `)`, `|`,
# `&`, `.` (6039 of the 6133). Requiring `{` or `:` keeps 95.8% of the
# declarations and drops 98.5% of the calls.
r"^[ \t]*(?:(?:public|private|protected|static|override|abstract|readonly)[ \t]+){0,4}(?:async[ \t]+)?(?:\*[ \t]*)?(?:get\s+|set[ \t]+)?(?!(?:if|for|while|switch|catch|return|throw|new|typeof|yield|await|void)\b|with\b[ \t\n]+\()(\[[^\]]+\]|[#]?[a-zA-Z_$][\w$]*)(?:[ \t\n]{0,50}<(?:[^<>]|<[^<>]*>)*>)?[ \t\n]{0,50}(\((?:[^()]|\((?:[^()]|\([^()]*\))*\))*\))[ \t\n]{0,50}[:{]|"
# #2773: the one declaration shape the `{`-or-`:` anchor above cannot
# cover. A constructor has no return type BY GRAMMAR, so a bodyless
# constructor overload signature (`constructor(runner: () => void, timeout:
# number);`, vscode/async.ts:1020) terminates in a bare `;` -- lexically
# identical to a call statement. Named explicitly rather than by loosening
# the anchor to accept `;`: all 125 declarations the anchor would otherwise
# lose on the crucible corpus are constructors, and accepting a bare `;`
# generally would readmit 4906 call statements.
r"^[ \t]*(?:(?:public|private|protected|abstract|declare)[ \t]+){0,3}(constructor)[ \t\n]{0,50}(\((?:[^()]|\((?:[^()]|\([^()]*\))*\))*\))[ \t\n]{0,50};",
re.M,
),
# 3. linear (Sequential Boundaries)
Expand Down
Loading
Loading