Skip to content

feat(scripts): check documented v3 method names against a committed OpenAPI snapshot #463

Description

@IgorShevchik

Measured 2026-09-02…04 against three portals: an on-premise build (SM_VERSION 26.700.0),
a cloud portal, and a disposable cloud sandbox. Every path:line below is from HEAD
6e8c85f.

What we claim

docs/content/docs/2.working-with-the-rest-api/7.discovering-v3-methods.md names the
portal's own document as the authority, starting with its own description: frontmatter:

Use rest.documentation.openapi to fetch the portal's own machine-readable list of every
available REST API v3 method — the source of truth the SDK relies on instead of a
hardcoded allowlist.

Nothing in the repository compares anything to that source of truth. The one gate that
looks at v3 drift says so itself — scripts/check-v3-method-refs.mjs:6-16:

NOTE: this script used to also check that v3 method names were on a hardcoded
version-manager allowlist. That allowlist was removed […] so there is nothing to
conform to and that layer was dropped.

It prints check-v3-method-refs: 0 problem(s), 0 warning(s), across docs, skills, README-AI (the wording since #450 moved it onto the shared reporter), correct by
its own definition: it checks action names (actions.v3.<x>), not portal method
names. That is what it was built for — its header cites #216, "guard against v3
method/action drift — phantom actions" — and it does that job. The gap is the other one:
#206, "validate/generate #supportMethods against the portal
/rest.documentation.openapi", closed without the snapshot it asked for.

What the portal does

await b24.actions.v3.call.make({ method: 'rest.documentation.openapi', params: {} })

HTTP 200, openapi: 3.0.0, Bitrix24 REST V3 API 1.0.0. Paths: 147 on-premise (8 tags
plus the untagged batch), 245 cloud, 220 sandbox.

Joining that document against every portal method name the repository mentions in a v3
context — docs/content/docs/, skills/, packages/jssdk/src/ including JSDoc
@example blocks, and packages/jssdk/README-AI.md:

on-premise
portal publishes 147
we name in a v3 context 25
overlap 8
we name, the portal does not publish 17
portal publishes, we never name 139
exercised live during the audit 21

Never named / total, per module: rest 41/42, note 23/25, tasks 16/19, main 3/5, and
four modules untouched end to end — humanresources 24/24, mail 24/24, call 4/4,
timeman 3/3. The *.field.get / *.field.list family alone is 50 of the 147.

Four of the 17 names in the other direction are defects, spread over eight occurrences,
every one in a file the current gate does not walk or does not inspect at that layer:

  • crm.contact.list — the method row of a v3 parameter table, in
    docs/…/2.call-list-rest-api-ver3.md:74 and 2.fetch-list-rest-api-ver3.md:72: "REST
    API method name that returns a data list (e.g., crm.contact.list, tasks.task.list)";
  • crm.item.get — packages/jssdk/src/core/actions/v3/call.ts:27 (CallV3 JSDoc: "REST
    API method name (eg: crm.item.get)") and the @example of v3/batch.ts:59-60;
  • crm.item.list — docs/…/5.filtering.md:179, a [v3]-tagged block calling
    actions.v3.callList.make({ method: 'crm.item.list', … }), and the same JSDoc line in
    v3/call-list.ts:41 and v3/fetch-list.ts:43;
  • tasks.task.comment.list — packages/jssdk/src/tools/batch-ref-v3.ts:49 @example;
    tasks publishes 19 methods on-premise and 22 in the cloud, none comment.*.

#464 fixes those eight by
hand; this one is the check that stops them coming back. If that one lands first, this
check should simply be silent.

An unknown v3 method fails softly: measured on main.eventlog.aggregate, a soft
METHODNOTFOUNDEXCEPTION. The message is portal-localised — this portal answered
Метод `main.eventlog.aggregate` не найден ("Method main.eventlog.aggregate not
found") — so nothing should match on that text.

Why it matters

A reader who copies the first example offered in a v3 parameter table gets
isSuccess === false and a "method not found" message, with no way to tell whether the
method is wrong or their portal is missing a module. That is the #206 class of defect,
caught today by a person reading the page, or not at all. Coverage is also unmeasured:
nobody reviewing a PR can say what share of the v3 surface the SDK describes, so a gap is
invisible until a user reports it. 139 of 147 is the size of the unknown.

The audit classified the 17 hits by hand, per occurrence, not per name. 13 of the names
are only ever used legitimately — the v2 selection matrix, limiter prose, the aggregate
survey in skills/VERIFICATION.md:54, the some.list / some.entity.aggregate
placeholders. A fourteenth, crm.item.get, is both a defect (v3/call.ts:27) and a
deliberate ❌ anti-pattern (skills/b24jssdk-rest/SKILL.md:407) — which is why the marker
convention below must work per line, not per name. Without a tool, that classification has
to be redone from scratch every time the docs move.

What to do

Grow scripts/check-v3-method-refs.mjs; do not add a tenth script. AGENTS.md lists "a
new script in scripts/" as a sign the process is generating work (#418); this is the same
subject, with the same wiring (pnpm run lint:v3-refs) and an existing lock spec. It
restores the layer its own NOTE says was dropped, without a hand-maintained list.

  1. Commit a reduced snapshot, not the raw document. On the wire the document is
    ~140 KB on-premise and ~215 KB cloud (139,977 / 219,914 bytes minified; pretty-printed
    on disk it is 356 KB / 576 KB), and its summary fields are Russian, which an
    English-only repository should not carry. The reduction the audit already writes is
    31 KB / 52 KB — { build, totals, modules, methodsPerModule, methods: [{ method, module, operations, requestFields, summary }] } — and dropping summary, the Russian part,
    makes the committed file smaller still. One per portal kind under e.g. scripts/data/,
    named by portal kind and snapshot date, never by domain, carrying no portal identity and
    no credential (the audit's reduction still keeps build.scopes; drop it). The step that
    talks to a portal stays separate and local.
  2. Add the method-name layer. Walk packages/jssdk/src/ alongside today's
    docs/content/docs, skills/, README-AI.md — five of the eight defect sites are
    TypeScript JSDoc. Count a name only in a method position (method: '…', call('…'),
    an OpenAPI path), decide v2/v3 context from a window of surrounding lines, and treat a
    bare backticked name in prose as informational: this repo writes result.items in
    backticks constantly. The v2 twins of those lines — v2/call.ts:26, v2/call-list.ts:30,
    v2/fetch-list.ts:31, v2/batch.ts:56-57, v2/batch-by-chunk.ts:48 — must stay silent;
    crm.* is correct there.
  3. Two verdict classes, one of which fails. The document is per portal, so a snapshot is
    a baseline, not a global truth. Measured: widening a webhook from 10 scopes to 15 did
    not change it at all, so it is not filtered by the token's rights; composition tracks
    installed modules and the build (inferred — we varied neither). So: a method-position
    name absent from every snapshot → failure with file:line; present in one snapshot
    only → informational, exit 0; a portal method we never name → informational, never a
    failure. That last is 139 of 147 today, and a check that is red by design gets disabled.
  4. A marker convention for deliberate anti-examples. Most mismatches are intentional,
    and the prototype hides them behind a hardcoded placeholder list, which rots. Use the
    convention the repo already applies to its ESLint rules: an inline marker next to the
    line, named in the failure message, "because a rule with no stated exit is one people
    work around silently" (AGENTS.md:183).
  5. Reporting and upkeep. A --coverage flag printing the method × snapshot ×
    documented table and the totals above, always exit 0 — that is what a maintainer or a
    docs PR cites. Document the refresh (which method, which reduction, when to re-run) in
    whichever .github/contributing/ guide you consider its home, bumping its
    Last reviewed stamp in the same PR.

A working offline prototype (claimed-surface.mjs plus coverage.mjs) produced every
number here; it is the shape of the join, not the deliverable.

What I would not do:

  • Not reintroduce a hardcoded allowlist of v3 methods. That went in 2.0.0, and
    b24pysdk shows why: its API_V3_METHODS holds 70 names, the on-premise build publishes
    81 it does not know and the cloud 179, 4 of its names exist on no portal, and a missing
    name silently downgrades the call to v2 even when the caller asked for v3. This tool
    audits prose against a snapshot; it must not gate runtime.
  • Not fail on undocumented portal methods (see 3). No two measured portals publish the
    same surface (147 / 245 / 220) — the same reason docs(v3): correct the documented page-size cap, add/update response shape, and batch ceiling #465
    argues against a checked-in per-method table. A snapshot is a baseline for prose, not a
    catalogue.
  • Not call a portal from CI. The snapshot keeps the check static; refreshing it stays a
    local step, like the rest of this repo's portal work.

Prior art, not a template to port: b24phpsdk commits docs/open-api/openapi.json, marks
classes with #[OpenApiEntity(entityKey: …)], and reports what is unmapped via
V3BuilderCoverageAuditor, filtered by module prefix. Their join is class →
components.schemas; ours would be method name in prose → paths.

How to verify

  1. Assuming the phantom-method corrections have not landed — if they have, the expected
    result is silence — node scripts/check-v3-method-refs.mjs should name the eight
    file:line sites above and stay silent on skills/b24jssdk-rest/SKILL.md:407 (the
    deliberate ❌ anti-pattern) and on the five v2 JSDoc lines in point 2. Both arms matter.
  2. --coverage against the committed on-premise snapshot reproduces 147 / 25 / 8 / 139 and
    the per-module table.
  3. The regression test belongs in scripts/__tests__/check-v3-method-refs.test.mjs, which
    already exists and runs under pnpm run docs:lint:test. One case per arm: fires on a
    name absent from every snapshot; silent on a marked anti-example, on a name present in
    one snapshot only, and on a backticked name in prose. Not jsSdk:unit — that project
    is test/integration/**/*.unit.spec.ts with no portal and no shared setup (axios is
    mocked per spec), and nothing here touches packages/jssdk/src/ runtime.
  4. Refreshing a snapshot end to end needs both pending rest.documentation.openapi fixes:
    without the first the call throws Cannot destructure property 'operating' of 'data' as it is undefined (that response carries no time block), and without the second
    getData() drops the document, whose body has no result envelope. With both, the call
    goes through the SDK unmodified.

Not verified

  • The check has never run in this repository's lint chain; its cost in CI is unmeasured.
  • The marker convention of point 4 has not been tried. The legitimate / defect split was
    made by reading each hit, not by a rule, so its false-positive rate is unknown.
  • Coverage was measured through an admin webhook only; whether an OAuth application
    receives a different document was not tested.
  • Each portal was snapshotted once, so staleness is a guess. What is measured: two
    different cloud portals, a day apart, disagree — 28 methods present on one and absent
    on the other, 6 with changed request fields (bizprocdesigner.template.get: id, select
    → templateId). Every on-premise method exists in the cloud; 98 cloud ones do not exist
    on the box.
  • b24phpsdk's auditor was read, not executed (its snapshot: 159 paths, 9 tags).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions