feat(parsing): add a CESR v1 stream parser - #402
Open
dhh1128 wants to merge 1 commit into
Open
Conversation
signify-ts has the primitive classes (Matter/Counter/Indexer) and EMITS framed
streams (eventing.messagize) but has no parser that CONSUMES a stream. Add
src/keri/core/parsing.ts: a deterministic v1 stream walker that frames messages
by version-string size + attachment counters (never by sniffing a leading '{',
so binary CBOR/MGPK bodies work too), delegating all primitive/counter/indexer
sizing to the existing classes. Every node carries byte-span provenance, and the
walk is resilient — on a code it cannot frame it stops and reports, keeping
everything parsed so far. Body decoders are pluggable (JSON built in). Exported
via src/exports.ts; 30 inline keripy-vector tests in test/core/parsing.test.ts.
Scope: CESR v1 only; the v2 count/genus/version-string machinery is a separate
follow-up.
Signed-off-by: Daniel Hardman <daniel.hardman@gmail.com>
Author
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #402 +/- ##
=======================================
Coverage 93.75% 93.75%
=======================================
Files 1 1
Lines 80 80
Branches 28 28
=======================================
Hits 75 75
Misses 3 3
Partials 2 2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
signify-ts has the CESR primitive classes (
Matter,Counter,Indexer) and canemit framed streams (
eventing.messagize), but there is no parser thatconsumes a stream — no equivalent of keripy's
keri.core.parsing.Parser. ThisPR adds one:
src/keri/core/parsing.ts, a deterministic v1 CESR stream walker.What it does
version-string size and the attachment counters — never by sniffing a
leading
{, so it works uniformly for JSON and (with an injected decoder) binaryCBOR/MGPK bodies.
Counter/Matter/Indexerclasses(their
{ qb64 }constructor +.code/.count/.qb64); it adds no code tables.{ start, end }) on every node, so callers canmap any decoded node back to the exact source bytes.
returning everything parsed so far (
{ messages, errors, consumed }) — a singlebad byte never discards the prefix that did parse.
-V/-0Vmaterial-quadletwrappers are treated as resilience boundaries (their self-declared size lets the
walk resume past undecodable inner content).
when a decoder for that serialization is injected via
ParseOptions. An undecodedbody is still framed (
sad: null).Public API:
parse(bytes, opts?),parseVersion(bytes, at), and the typed outputcontract (
ParseResult,CesrMessage,AttachmentGroup,Primitive,ParseError, …). Wired into the barrel via one line insrc/exports.ts.No new dependencies
This PR adds zero dependencies — runtime or dev.
package.jsonand the lockfileare untouched.
parsing.tsimports only./counter.ts,./indexer.ts,./matter.ts(modules signify-ts already ships) plus JS builtins (TextDecoder,JSON). It reuses the primitive classes rather than adding anything.Excludable for consumers who don't want it
The module is side-effect-free: it does no work at import time (the shared
TextDecoderis created lazily on first use, not at module load). So a consumerusing named imports (
import { Matter } from 'signify-ts') with anytree-shaking bundler drops
parsing.jsentirely when the parser is unused. If itis included, the footprint is small — pure JS, no transitive deps.
Two caveats / optional follow-ups (maintainers' call — deliberately not in this PR):
export default exp(namespace) insrc/index.tsdefeatstree-shaking for anyone using the default import; named imports are unaffected.
"sideEffects": falsetopackage.jsonand/or a subpath export(
signify-ts/parsing) would make exclusion first-class, but those are repo-widepackaging decisions to validate separately.
Testing
test/core/parsing.test.ts— 30 tests, inline keripy-derived vectors (matching theexisting house convention), covering: version parsing;
-A/-B/-C/-D/-E/-F/-G/-H/-Igroups;
-V/-0Vnested decomposition and resilience boundaries; the Matter vsIndexer primitive-class discriminator; and every error path (no-version-string,
malformed-body, unparseable-counter, unframable-group).
Gates:
npm run build,npm run lint,npm run pretty:check, andnpx vitest run test/core/parsing.test.tsall pass. (The 4 unrelated failures intest/app/*require a running KERIA agent and are pre-existing onmain.)Scope
CESR v1 only. signify-ts's counter/version-string machinery is v1 today (no v2
count-code table, no genus/version model beyond the opaque
--AAAcounter,Serialsis JSON-only,deversify/VEREXare locked to the 17-char v1 versionstring), and this parser matches that. The v2 additions (v2 count codes,
genus/version, a v2 version-string parser) are a separate follow-up so this PR stays
a self-contained, reviewable unit.