Skip to content

feat(parsing): add a CESR v1 stream parser - #402

Open
dhh1128 wants to merge 1 commit into
WebOfTrust:mainfrom
dhh1128:feat/cesr-stream-parser
Open

feat(parsing): add a CESR v1 stream parser#402
dhh1128 wants to merge 1 commit into
WebOfTrust:mainfrom
dhh1128:feat/cesr-stream-parser

Conversation

@dhh1128

@dhh1128 dhh1128 commented Jul 21, 2026

Copy link
Copy Markdown

Summary

signify-ts has the CESR primitive classes (Matter, Counter, Indexer) and can
emit framed streams (eventing.messagize), but there is no parser that
consumes a stream — no equivalent of keripy's keri.core.parsing.Parser. This
PR adds one: src/keri/core/parsing.ts, a deterministic v1 CESR stream walker.

What it does

  • Frames a byte stream into messages + typed attachment groups, by the
    version-string size and the attachment counters — never by sniffing a
    leading {, so it works uniformly for JSON and (with an injected decoder) binary
    CBOR/MGPK bodies.
  • Delegates all sizing to the existing Counter / Matter / Indexer classes
    (their { qb64 } constructor + .code/.count/.qb64); it adds no code tables.
  • Carries byte-span provenance ({ start, end }) on every node, so callers can
    map any decoded node back to the exact source bytes.
  • Is resilient: on a code it cannot frame it stops and reports a typed error,
    returning everything parsed so far ({ messages, errors, consumed }) — a single
    bad byte never discards the prefix that did parse. -V/-0V material-quadlet
    wrappers are treated as resilience boundaries (their self-declared size lets the
    walk resume past undecodable inner content).
  • Pluggable body decoders: JSON is built in; CBOR/MGPK etc. are decoded only
    when a decoder for that serialization is injected via ParseOptions. An undecoded
    body is still framed (sad: null).

Public API: parse(bytes, opts?), parseVersion(bytes, at), and the typed output
contract (ParseResult, CesrMessage, AttachmentGroup, Primitive,
ParseError, …). Wired into the barrel via one line in src/exports.ts.

No new dependencies

This PR adds zero dependencies — runtime or dev. package.json and the lockfile
are untouched. parsing.ts imports only ./counter.ts, ./indexer.ts,
./matter.ts (modules signify-ts already ships) plus JS builtins (TextDecoder,
JSON). It reuses the primitive classes rather than adding anything.

Excludable for consumers who don't want it

The module is side-effect-free: it does no work at import time (the shared
TextDecoder is created lazily on first use, not at module load). So a consumer
using named imports (import { Matter } from 'signify-ts') with any
tree-shaking bundler drops parsing.js entirely when the parser is unused. If it
is included, the footprint is small — pure JS, no transitive deps.

Two caveats / optional follow-ups (maintainers' call — deliberately not in this PR):

  • The barrel's export default exp (namespace) in src/index.ts defeats
    tree-shaking for anyone using the default import; named imports are unaffected.
  • Adding "sideEffects": false to package.json and/or a subpath export
    (signify-ts/parsing) would make exclusion first-class, but those are repo-wide
    packaging decisions to validate separately.

Testing

test/core/parsing.test.ts — 30 tests, inline keripy-derived vectors (matching the
existing house convention), covering: version parsing; -A/-B/-C/-D/-E/-F/-G/-H/-I
groups; -V/-0V nested decomposition and resilience boundaries; the Matter vs
Indexer primitive-class discriminator; and every error path (no-version-string,
malformed-body, unparseable-counter, unframable-group).

Gates: npm run build, npm run lint, npm run pretty:check, and
npx vitest run test/core/parsing.test.ts all pass. (The 4 unrelated failures in
test/app/* require a running KERIA agent and are pre-existing on main.)

Scope

CESR v1 only. signify-ts's counter/version-string machinery is v1 today (no v2
count-code table, no genus/version model beyond the opaque --AAA counter,
Serials is JSON-only, deversify/VEREX are locked to the 17-char v1 version
string), and this parser matches that. The v2 additions (v2 count codes,
genus/version, a v2 version-string parser) are a separate follow-up so this PR stays
a self-contained, reviewable unit.

signify-ts has the primitive classes (Matter/Counter/Indexer) and EMITS framed
streams (eventing.messagize) but has no parser that CONSUMES a stream. Add
src/keri/core/parsing.ts: a deterministic v1 stream walker that frames messages
by version-string size + attachment counters (never by sniffing a leading '{',
so binary CBOR/MGPK bodies work too), delegating all primitive/counter/indexer
sizing to the existing classes. Every node carries byte-span provenance, and the
walk is resilient — on a code it cannot frame it stops and reports, keeping
everything parsed so far. Body decoders are pluggable (JSON built in). Exported
via src/exports.ts; 30 inline keripy-vector tests in test/core/parsing.test.ts.

Scope: CESR v1 only; the v2 count/genus/version-string machinery is a separate
follow-up.

Signed-off-by: Daniel Hardman <daniel.hardman@gmail.com>
@dhh1128

dhh1128 commented Jul 21, 2026

Copy link
Copy Markdown
Author

tagging @kentbull and/or @m00sey for review. Kent, this may be redundant with what you were planning; if so, I won't be offended if you just close this.

@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.75%. Comparing base (9d35b06) to head (4c9aab4).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #402   +/-   ##
=======================================
  Coverage   93.75%   93.75%           
=======================================
  Files           1        1           
  Lines          80       80           
  Branches       28       28           
=======================================
  Hits           75       75           
  Misses          3        3           
  Partials        2        2           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant