Skip to content

feat(scripts): hold v3 method names against a portal's own OpenAPI document - #471

Merged
IgorShevchik merged 2 commits into
mainfrom
feat/v3-method-coverage-snapshot
Sep 5, 2026
Merged

IgorShevchik merged 2 commits into
mainfrom
feat/v3-method-coverage-snapshot

Conversation

@IgorShevchik

Copy link
Copy Markdown
Collaborator

Closes #463. Closes #464 — see Why both below.

The gap

The discovery page calls rest.documentation.openapi "the source of truth the SDK relies on
instead of a hardcoded allowlist", and nothing compared anything to it. The one gate that
looks at v3 drift said so in its own NOTE: the method layer went out with the allowlist in
2.0.0, and #206 was closed without the snapshot it asked for.

The check

scripts/data/openapi-cloud-2026-09-05.json is a reduction of a real portal's document —
method name, module, operations. A name used in a v3 method position that no snapshot
publishes fails with file:line.

Three properties are load-bearing, each a decision rather than a default:

  • A snapshot is a baseline, not a catalogue. Portals differ — two cloud portals measured a
    day apart disagreed on 28 methods; every on-premise method exists in the cloud, 98 cloud
    ones do not exist on the box. So one snapshot publishing a name is enough, and a portal
    method we never document is informational (237 of 245 today). A check that is red by design
    gets switched off.
  • It audits prose, never runtime. b24pysdk keeps a 70-name list and silently downgrades
    an unlisted v3 call to v2. Not repeating that.
  • It never calls a portal. --refresh is local, and a plain POST rather than an SDK
    import — every other script here is dependency-free node, and importing the package would
    drag the Pull client's protobuf modules into a lint script.

What counts as a method position is the whole difficulty, because this repo writes
result.items and crm.item.list in backticks identically. Four positions, each taken from a
real occurrence: a method: literal, a v3 batch tuple, a parameter-table row whose first cell
is method, and a JSDoc bullet documenting the method option. The last is why
packages/jssdk/src/ joined the walk — five of the defects were JSDoc, not docs.

What it found

The eight sites #463 predicted, at the predicted lines — and a ninth it did not:
5.umd.md offers a newcomer actions.v3.call.make({ method: 'crm.item.list' }) as their
first copy-paste, and v3 publishes no crm.item.* on any portal.

Why both issues

All ten occurrences are fixed here rather than marked, because a check that cannot be switched
on is not a deliverable. That is #464's scope, so it closes too. The substantive one was
5.filtering.md's "same query in v2 and v3" section: it cannot be the same method, so the
v3 half now asks the same shape of question of tasks.task.list, with field names read
from tasks.task.field.list rather than guessed.

Two findings from building it

A shared escape hatch is not an escape hatch. Two fences already carried
// @check-ignore: top-level return in error-handling illustration, written for the typecheck
gate about something else — and my first version let them silence this gate too. The marker's
reason must now name the method it excuses, so an exemption granted for one reason cannot
acquire a second by sitting in the right place.

The anti-example in b24jssdk-rest/SKILL.md was silent by accident, not intent: its
neighbourhood mentions both actions.v2. and actions.v3., so the dialect is ambiguous and
the line is skipped. Defensible as a rule; not something to rely on. Marked.

Tests

Eight new cases in scripts/__tests__/check-v3-method-refs.test.mjs (12 total), one per arm.
Confirmed by mutation, including the fence-tag rule — which at first no test could
distinguish from its own fallback
, so a case was added where it is the only signal.

mutation red
marker no longer has to name the method 1
fence [v2]/[v3] tag ignored 1

Coverage

pnpm run lint:v3-coverage, always exit 0 — the number a docs PR quotes:

publishes 245, documented here 8, never named 237
  humanresources  0 / 33      mail  0 / 24      bizprocdesigner  0 / 16
  tasks           4 / 22      main  2 / 37      rest             1 / 55

The snapshot

33 KB of the 215 KB on the wire. Carries a portal kind, never a domain, a scope or a
credential; the Russian summary fields are dropped. Verified: no host, no token, no URL in
the file. Refresh is documented in .github/contributing/documentation.md.

Checks

  • pnpm typecheck, pnpm lint, pnpm lint:md — clean
  • docs-lint --strict — 0/0 (the @check-ignore threshold is raised 51 → 57 with the reason
    written where the previous raises are)
  • docs:lint:test — 155 pass
  • all four block gates, docs:lint-api-index — clean
  • pnpm vitest run --project jsSdk:unit — 609 passed

🤖 Generated with Claude Code

https://claude.ai/code/session_01F22e2ft66y7nuBJjzdThBr


Generated by Claude Code

…cument

The discovery page calls `rest.documentation.openapi` "the source of truth the
SDK relies on instead of a hardcoded allowlist", and nothing compared anything
to it. The one gate that looks at v3 drift said so in its own NOTE: the method
layer went out with the allowlist in 2.0.0, and #206 was closed without the
snapshot it asked for.

It is back, and deliberately not as a list. `scripts/data/openapi-cloud-*.json`
is a reduction of a real portal's document -- method name, module, operations --
and a name used in a v3 method position that no snapshot publishes now fails
with `file:line`.

Three properties are load-bearing, and each is a decision rather than a default:

A snapshot is a **baseline, not a catalogue**. Portals genuinely differ: two
cloud portals measured a day apart disagreed on 28 methods, and every on-premise
method exists in the cloud while 98 cloud ones do not exist on the box. So one
snapshot publishing a name is enough, and a portal method we never document is
informational -- 237 of 245 today. A check that is red by design gets switched
off.

It audits **prose, never runtime**. `b24pysdk` keeps a 70-name list and silently
downgrades an unlisted v3 call to v2; the box publishes 81 names it does not
know and 4 of its own exist on no portal at all. That is the mistake not to
repeat.

It **never calls a portal**. `--refresh` is a local step, and it is a plain POST
rather than an SDK import: every other script here is dependency-free node, and
pulling the package in would drag the Pull client's protobuf modules into a lint
script. The snapshot carries a portal *kind* -- never a domain, a scope or a
credential; 33 KB of the 215 KB the wire carries, with the Russian `summary`
fields dropped.

What counts as a method position is the whole difficulty, because this
repository writes `result.items` and `crm.item.list` in backticks identically.
Four positions, every one taken from a real occurrence: a `method:` literal, a
v3 batch tuple, a parameter-table row whose first cell is `method`, and a JSDoc
bullet documenting the `method` option. The last is why `packages/jssdk/src/`
joined the walk -- five of the defects were JSDoc, not documentation.

The check found the eight sites the issue predicted, at the predicted lines, and
a ninth it did not: `5.umd.md` offers a newcomer `actions.v3.call.make({ method:
'crm.item.list' })` as their first copy-paste, and v3 publishes no `crm.item.*`
on any portal. All ten occurrences are fixed here rather than marked, because a
check that cannot be switched on is not a deliverable -- which also closes #464.
`5.filtering.md`'s "same query in v2 and v3" section was the substantive one: it
could not be the same *method*, so the v3 half now asks the same shape of
question of `tasks.task.list`, with field names read from
`tasks.task.field.list` rather than guessed.

Two findings from building it are worth keeping:

**A shared escape hatch is not an escape hatch.** Two fences already carried
`// @check-ignore: top-level return in error-handling illustration`, written for
the typecheck gate about something else entirely -- and my first version let
them silence this gate too. The marker's reason must now name the method it
excuses, so an exemption granted for one reason cannot acquire a second by
sitting in the right place.

**The anti-example in `b24jssdk-rest/SKILL.md` was silent by accident**, not by
intent: its neighbourhood mentions both `actions.v2.` and `actions.v3.`, so the
dialect is ambiguous and the line is skipped. That is a defensible rule, but
relying on it is not, so the line is marked.

Both new branches are confirmed by mutation, including the fence-tag rule, which
at first no test could distinguish from its own fallback.

Coverage, always exit 0, is what a docs PR quotes: 245 published, 8 documented,
237 never named -- `humanresources` 0/33, `mail` 0/24, `bizprocdesigner` 0/16
untouched end to end.

Closes #463
Closes #464
Refs #113, #206, #216

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F22e2ft66y7nuBJjzdThBr
The Engineer found the one that mattered: `METHOD_NAME` demanded lower-case
throughout, so it could not see `crm.activity.mail.getContent` or
`crm.activity.mail.getThread` -- both published by the very snapshot this gate
validates against. A name it cannot recognise is not reported as wrong; it is
silently not seen, which is the worse failure for a check whose whole claim is
fidelity to the real surface. Segments may be camelCase now.

QA found three rules that no mutation could reach: the batch tuple, the
parameter-table row, and the path fallback -- the last because the test meant to
pin it put `actions.v3.` within the nearby-lines window, so the cheaper rule
answered first. All three have a case now, each confirmed by deleting the rule.

QA also surfaced a real bug through its own test harness. The path rule matched
`v3` as a *substring*, and `mkdtempSync` names its directories `v3-refs-…`, so
the harness classified every fixture as v3 regardless of where the file sat. In
this repository that would mean a directory called `v3-migration-notes/` forcing
v3 onto a page about v2. Matched by path segment now, and the mutation back to a
substring reddens.

`/code-review` found two more. A marker inside a `.ts` JSDoc block trims to
`* // @check-ignore`, so the fence form never matched there at all -- only the
line-above form did, which is why `aggregate.ts` needed its marker adjacent. And
a marked placeholder was still counted as documented, so `--coverage` re-listed
it under "published by no snapshot", inflating the number the report exists to
give. Coverage now reads 8 named, not 11.

Documentation caught the two edits I got wrong, both mine:

The UMD page's `#rest-api-ver3` tab called `actions.v2.call.make`. Fixing the
method name by changing the *action* left a version-switcher whose v3 tab
demonstrated v2 -- correct code, useless page. It loads tasks through
`actions.v3.call.make` now, with field names read from `tasks.task.field.list`
rather than guessed; `created`, not the `createdDate` I first wrote.

The `aggregate.ts` marker sat inside the public `@example`, so an SDK consumer
would read `// @check-ignore` on hover as part of the example. It is above the
tag now, and a marker anywhere in the enclosing JSDoc block exempts that block --
which is how a reader would expect a block annotation to behave anyway.

Security found one, low and local: an uncaught network error in `--refresh`
prints Node's `cause`, which carries `getaddrinfo ENOTFOUND <hostname>`. The
secret is in the path rather than the host, but neither belongs in a terminal
someone may paste from. Caught and discarded.

Smaller, from the same pass: a truncated snapshot crashed from inside a `.map`
with nothing naming the file; `~~~` is a valid CommonMark fence and both scans
were backtick-only; `callMethod('x.y')` was an unread position, present in the
migration pages today; and the script ran its body on import, so importing
`reduceDocument` for a test ended the run instead.

The CTO asked what keeps the snapshot current. Nothing does, and that is #472
rather than something to invent here -- the gate is asymmetric on purpose (any
one snapshot suffices, an undocumented portal method never fails), so a stale
snapshot costs a false failure with a message that says how to clear it, and
never a silent pass.

Refs #463, #464, #472

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F22e2ft66y7nuBJjzdThBr
@IgorShevchik
IgorShevchik merged commit 36dd2e6 into main Sep 5, 2026
10 checks passed
@IgorShevchik
IgorShevchik deleted the feat/v3-method-coverage-snapshot branch September 6, 2026 03:33
IgorShevchik added a commit that referenced this pull request Sep 9, 2026
`crm.item.list` is not a `restApi:v3` method — the example on the v3 page was
generated from the v2 one with only the action namespace swapped. Caught by
`check-v3-method-refs`, which the `docs-lint` CI job runs and `pnpm docs:lint`
does not: the local script is `pnpm run lint:v3-refs`, and it is the gate #471
added for exactly this mistake.

Replaced with `main.eventlog.list`, which the committed snapshot publishes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants