Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugins/knowledge/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
"name": "knowledge",
"version": "0.12.1",
"version": "0.12.2",
"description": "Ingest external knowledge into durable, synthesized artifacts. Ships a book-distillation pipeline (PDF/EPUB into concept-organized, author-attributed skill reference files), a YouTube pipeline (watch, transcript, link harvest, and repo-applicability synthesis), a course-digest pipeline (extract and synthesize online video courses \u2014 Dometrain, Teachable \u2014 into repo-applicable recommendations), a docpage-digest pipeline (single online documentation page into a verified knowledge slice with dual verification \u2014 one cross-vendor verifier \u2014 and an interview handoff), and a map-corpus pipeline (multi-resource corpus into a classified link map, deterministic node manifests, gate-verified relevance inventory, and an approved queue of docpage-digest runs), plus a re-runnable setup action; a configurable library directory governs where synthesized artifacts land in the consuming repo.",
"author": {
"name": "Melodic Software",
Expand Down
15 changes: 15 additions & 0 deletions plugins/knowledge/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,21 @@ All notable changes to the `knowledge` plugin are recorded here. The `version` i
`.claude-plugin/plugin.json` is the delivery vehicle — a consumer receives a change
only after that version increases.

## [0.12.2]

### Changed

- **`map-corpus` states its own deferred decisions instead of pointing outside itself.** The skill
cited an authoring-time planning document by label, which no consumer ever receives — an
unresolvable reference on a shipped surface. Each site is now self-contained: the deferred rung-3
decision states its own fork (a presence-gated `/firecrawl:firecrawl map` seam versus a recorded
reimplementation), its user-reserved arbiter, and its trigger; the deferred repo-tree enumeration
rung states its trigger; the opaque `Q19` label is dropped from `SKILL.md`,
`discovery/link-map-format.md`, `discovery/check_linkmap.py`, and the eval set; and the
whole-snapshot hash in `extraction/node-manifest-format.md` now points at
`reference/citation-shape.md`, its actual owner. No behavior, schema, gate, exit code, or
argument changes.

## [0.12.1]

### Added
Expand Down
15 changes: 9 additions & 6 deletions plugins/knowledge/skills/map-corpus/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,13 +89,16 @@ Seeds come in two kinds, told apart by their normalized path:
snapshot into `discovery/`: **rung 1** `<origin>/llms.txt`; **rung 2** the sitemap
(`sitemap.xml`, a markdown variant such as `sitemap.md` when the site serves one, or the
location robots.txt declares). An origin seed where **neither rung resolves → stop loudly**:
rung 3 (in-page link extraction) is deliberately absent, and its design — a presence-gated
`/firecrawl:firecrawl map` seam versus a recorded reimplementation — is a user-reserved
decision (Brief Q19) triggered by exactly this stop.
rung 3 (in-page link extraction) is deliberately absent. Its design is a **user-reserved**
decision — a presence-gated `/firecrawl:firecrawl map` seam with a documented in-skill fallback,
versus a recorded reason this skill reimplements in-page extraction (a bare unguarded
cross-plugin reference is barred, and `dependencies` are reserved for hard requires). This stop
IS that decision's trigger: report it and ask, never crawl unasked.
- **Resource seed** (non-root path, e.g. a raw repo file URL) — itself a corpus resource: a
link-map row with rung `seed`, no discovery at its origin. Human-enumerated resource seeds are
the V1 ingress for repository files (a repo-tree enumeration rung is deferred, recorded in the
Brief). **V1 requires at least one origin seed** — the link-map gate needs a discovery basis;
the V1 ingress for repository files; a repo-tree enumeration rung (`git ls-tree` / tree API) is
deferred, and its trigger is the first corpus whose repository half is too large to enumerate by
hand. **V1 requires at least one origin seed** — the link-map gate needs a discovery basis;
a resource-seeds-ONLY corpus is outside V1 mapper scope (recorded deferral in
`discovery/link-map-format.md`): route those URLs to direct `/knowledge:docpage-digest` runs.
**GitHub blob URLs:** seed the `raw.githubusercontent.com` form — a `blob` URL snapshots the
Expand Down Expand Up @@ -178,7 +181,7 @@ Emit a continuation prompt when pausing mid-pipeline (slug, first unticked phase
## What this skill does NOT do

- **Does not digest.** Verdicts and evidence tokens, yes; digests are `docpage-digest`'s job.
- **Does not crawl in-page links.** Discovery is rungs 1–2; rung 3 is user-reserved (Q19).
- **Does not crawl in-page links.** Discovery is rungs 1–2; rung 3 is user-reserved (see Phase 1).
- **Does not commit or graduate.** The slice is untracked and self-ignoring.
- **Does not route non-web ingest types.** A YouTube/course/book URL discovered in the corpus
is classified `companion` for the interview, never dispatched to sibling pipelines.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ def main(argv=None) -> int:
fail(2, "at least one --discovery output is required: a link map with "
"no discovery basis cannot demonstrate coverage. Two cases "
"land here: (1) an origin seed resolved neither llms.txt nor "
"a sitemap -- stop loudly (rung 3 is deferred by Q19); "
"a sitemap -- stop loudly (rung 3 is deferred, user-reserved); "
"(2) a corpus of resource seeds only -- outside V1 gate scope "
"by recorded deferral (see link-map-format.md); route those "
"URLs to direct docpage-digest runs instead.")
Expand Down
16 changes: 9 additions & 7 deletions plugins/knowledge/skills/map-corpus/discovery/link-map-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ of what it saw. An agent classifies; it does not get to choose what needs classi
user says so), snapshot it into the slice under `discovery/`.
2. **Rung 2 — sitemap**: fetch the site's sitemap (`sitemap.xml`, `sitemap.md`, or a
robots.txt-declared location), snapshot likewise.
3. **Rung 3 — in-page link extraction: NOT IN V1.** Deferred question Q19 (Brief): reaching it
3. **Rung 3 — in-page link extraction: NOT IN V1.** Deferred and USER-RESERVED: reaching it
requires either a presence-gated `/firecrawl:firecrawl map` seam or a recorded reason to
reimplement — USER-RESERVED. The trigger is the first corpus whose seeds resolve neither an
reimplement. The trigger is the first corpus whose seeds resolve neither an
`llms.txt` nor a sitemap. Until then a corpus with neither artifact stops loudly at discovery.

Fetching is the skill's job (WebFetch/curl per the skill's own text, channel recorded); parsing
Expand Down Expand Up @@ -91,7 +91,8 @@ nobody parsed.
- `rows` — exactly one row per distinct URL across seeds + every discovery output. Each row:
- `url` — normalized URL.
- `rungs` — non-empty subset of `seed` | `llms-txt` | `sitemap-xml` | `sitemap-md`, the
provenance of every appearance. (`in-page` joins this enum only when Q19 is decided.)
provenance of every appearance. (`in-page` joins this enum only when the deferred rung-3
decision above is made.)
- `classification` — exactly one of:
- `in-corpus` — fetched, snapshotted, node-extracted, inventoried, queued for digestion.
- `companion` — same corpus context but a different ingest type (e.g. a repo, a video);
Expand All @@ -111,12 +112,13 @@ same recorded deferral as the repo-tree enumeration rung). Checks:
1. All inputs parse (duplicate JSON keys rejected at any depth); unknown or missing keys rejected
in the link map, its rows, AND each discovery output; schema literals exact; seeds and row
URLs must be in normalized form. Failures name the file/row; exit 2 for unusable input.
2. **Classification coverage** (Brief criterion 2): every URL in every discovery output and every
seed has exactly one row; every row's URL traces back to at least one discovery output or the
seed list (no phantom rows); every row carries a valid classification and non-empty reason;
2. **Classification coverage** — this gate's reason to exist: every URL in every discovery output
and every seed has exactly one row; every row's URL traces back to at least one discovery
output or the seed list (no phantom rows); every row carries a valid classification and
non-empty reason;
every row's `rungs` match where the URL actually appeared, exactly.
3. **Bounds**: `in-corpus` row count ≤ `bounds.max_resources`, else a named failure — the
bound-breach stop that forces the run back to the user (Brief criterion 6).
bound-breach stop that forces the run back to the user.
4. A clean run prints what it exercised (files, row/URL counts, per-classification tally).

Exit codes: 0 pass; 1 named check failures; 2 unusable input; 3 internal gate bug.
Expand Down
4 changes: 2 additions & 2 deletions plugins/knowledge/skills/map-corpus/evals/evals.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@
"id": 3,
"name": "no-discovery-artifacts-stops-loudly",
"prompt": "Run /knowledge:map-corpus \"internal wiki\" https://wiki.internal.example where the site serves neither llms.txt nor any sitemap.",
"expected_output": "Stops loudly at discovery: reports that neither rung 1 (llms.txt) nor rung 2 (sitemap) resolved, names the deferred rung-3 decision (Q19, user-reserved firecrawl seam vs reimplementation), and asks the user rather than crawling in-page links.",
"expected_output": "Stops loudly at discovery: reports that neither rung 1 (llms.txt) nor rung 2 (sitemap) resolved, names the deferred rung-3 decision (user-reserved: a presence-gated firecrawl map seam versus a recorded reimplementation), and asks the user rather than crawling in-page links.",
"expectations": [
"Does not extract in-page links as a fallback",
"Names the Q19 deferral and its user-reserved arbiter",
"Names the deferred rung-3 decision and that its arbiter is the user",
"Does not silently classify the corpus as empty"
]
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ agent chose to enumerate.
machine paths, locale-dependent text, or unordered collections enter the output. The snapshot is
recorded by basename only.
3. **Self-verifying spans.** `content_sha256` is SHA-256 over the raw snapshot bytes
`[start_byte, end_byte)`. `snapshot.sha256` is SHA-256 over the whole snapshot as fetched (the
Brief's captured content-hash assumption).
`[start_byte, end_byte)`. `snapshot.sha256` is SHA-256 over the whole snapshot as fetched — the
hash the tracked citation shape carries (`${CLAUDE_PLUGIN_ROOT}/reference/citation-shape.md`).
4. **Document order.** `nodes` is ordered by `start_byte`; `index` is the 0-based position.
5. **Fail loudly.** Empty snapshot, unreadable file, unknown extension, non-UTF-8 BOM, CR-only
line endings, or a partition self-check failure exits non-zero with a message. The extractor
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Emitters must never emit a duplicate key.
- `quote` — non-empty string, verbatim from the snapshot.
- `start_byte` / `end_byte` — the quote's exact span in RAW SNAPSHOT bytes (end exclusive).

## Evidence-token byte mapping (the rule the Brief owed step 2)
## Evidence-token byte mapping

The manifest is byte-addressed; quotes are text. The mapping rule:

Expand Down Expand Up @@ -94,5 +94,5 @@ Exit codes: 0 all checks passed; 1 one or more named check failures; 2 unusable
(parse/IO/schema-literal errors); 3 internal invariant violation. A clean run prints what it
exercised (file names, row count, node count, field list) — silence is never a pass, and a pass
covers only what was printed. The gate was written and tested to fail loudly on unparsable and
malformed input BEFORE being made a required artifact (Brief acceptance criterion 4; two prior
gates in this codebase shipped fail-open and were caught by verifiers).
malformed input BEFORE being made a required artifact (two prior gates in this codebase shipped
fail-open and were caught by verifiers).
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
#!/usr/bin/env python3
"""Tests for check_inventory.py. Stdlib unittest; run: python test_check_inventory.py

The fail-loud cases here are the Brief's acceptance criterion 4: this gate is
proven to fail loudly on unparsable and malformed input BEFORE it is made a
required artifact.
The fail-loud cases here are why this file exists: the gate is proven to fail
loudly on unparsable and malformed input BEFORE it is made a required artifact.
"""

import copy
Expand Down