Knowledge Compiler

Auditable structured-evidence infrastructure with partial provenance recovery.

Knowledge Compiler transforms textbook-derived records into Knowledge IR—a typed, backend-independent intermediate representation with automated structural checks.

Source → Knowledge IR → Target Backend (YAML / Neo4j / RDF / Skill / MCP)

Most AI systems that use textbooks do one of two things:

• RAG — retrieve chunks of raw text. Fast but unstructured. No guarantees on what the model will find.
• Fine-tuning — expensive, opaque, hard to update.

This is a third engineering path: compile records into a deterministic, composable intermediate representation that an agent can query directly. Source traceability is recoverable for a subset; scientific validity is not inferred from structure.

Historical inventory: ACSM 12th Ed. (707 files) + NSCA-CSCS 5th Ed. (1,598 files) Total: 2,305 files representing 2,294 unique IDs; 571 records meet the strongest recovered-subset criterion

---

What can it do right now?

Ask it anything from the textbook:

Q: "Can a 55-year-old man with hypertension begin vigorous exercise?"

The system loads:
  → threshold.cvd_risk_factors    (age ≥45 = risk factor, BP ≥130/80 = risk factor)
  → procedure.acsm_screening      (non-exerciser + 2+ risk factors + vigorous = clearance needed)
  → recommendation.medical_clearance

Answer: Medical clearance is recommended before starting vigorous exercise.
Sources: ACSM12 Ch2.2.11 (p.150), Ch2.2.7 (p.142)

This example illustrates a typed lookup with stored page locators. Locator presence does not by itself establish source fidelity, factual correctness, or scientific validity.

---

Quick demo

from knowledge_compiler import Skill

# Load compiled Knowledge IR
skill = Skill("sources/books/acsm12")

# Safety check — what signs suggest I should see a doctor?
warnings = skill.get("warning.signs_symptoms_cv_disease")
# →
# {
#   "type": "Warning",
#   "signs": [
#     "Pain or discomfort in chest/neck/jaw/arms",
#     "Shortness of breath at rest or mild exertion",
#     "Dizziness or syncope",
#     ...
#   ],
#   "action": "Seek medical evaluation before initiating exercise",
#   "severity": "high",
#   "source": "ACSM12 Ch2.4 (p.133)"
# }

# Threshold check
bmi = skill.get("threshold.bmi_classification")
# → BMI 27 → "Overweight (25.0-29.9)"

# Procedure lookup
protocol = skill.get("procedure.one_rm_testing")
# → 8 steps, from warm-up to failure

Every direct lookup returns a stored typed object, not newly generated prose. Structural checks and stored citation fields do not make every object source-verified or scientifically valid.

---

Coverage

2,305 historical files · 2,294 unique IDs · 837 edges · 13 types

By Book

Book	Historical files	Types	Repository chapter directories	Status
ACSM 12th Ed.	707	13	12	✅ Released
NSCA-CSCS 5th Ed.	1598	13	26	✅ Released
Total	2,305 files / 2,294 unique IDs	13	38	Structurally queryable historical inventory

By Type

Type	ACSM	NSCA	Total	Example
Concept	141	918	1059	Exercise, VO2max, Sarcomere
Threshold	206	114	320	BMI, MET ranges, BP
Recommendation	139	174	313	PA guidelines, training
TableRow	127	124	251	Normative data, references
Procedure	29	98	127	1-RM testing, screening
Warning	34	46	80	SCD risk, overtraining
Evidence	2	82	84	Research findings
Formula	5	23	28	Cooper, Siri equation
Figure	16	7	23	Screening algorithms
Contraindication	4	1	5	Exercise CI
Table	2	3	5	Reference tables
RiskFactor	1	3	4	CVD risk factors
DecisionRule	1	2	3	Screening decisions

Historical Development Comparison

Historical-only: This 10-question development comparison has no preserved executable scorer/adjudication lineage and used unequal search spaces. It is not a validated benchmark or evidence of system superiority.

Condition	Questions	Accuracy	95% CI	Sample limitation
Knowledge Compiler	10	100%	[69%, 100%]	Small sample; one question shifts 10pp
RAG on PDF (MiniLM)	10	38%	[14%, 68%]	Same limitation
Raw LLM (DeepSeek)	10	44%	[19%, 73%]	Same limitation

The table is retained as development history. Its percentages must not be interpreted as current accuracy or comparative-performance evidence.

Scope note: v0.1 was structurally evaluated on ACSM 12th Ed., Chapters 1–3. The ontology and pipeline are designed to be source-extensible, but cross-domain generalization (e.g., chemistry, law) has not yet been tested.

---

Why Knowledge IR?

Knowledge Compiler exposes several implementation properties that differ from RAG and fine-tuning. This is a conceptual engineering comparison, not a validated performance comparison:

Problem	RAG	Fine-tuning	Knowledge IR
Determinism	❌ Same query, different answer	❌ Same input, same weights	✅ Same query, same objects, same result
Stored locators	Chunk metadata	Source not explicit at inference	Page/span fields when present; fidelity varies by record
Composability	❌ Retrieval only	❌ Static	✅ Object → Agent → Workflow
Maintainability	❌ Re-embed entire corpus	❌ Re-train entire model	✅ Add/remove one object

Direct object lookup is deterministic for a fixed store. No universal guarantee is made about source fidelity, factual correctness, or downstream answer quality.

---

How it's different

	RAG	Fine-tuning	RAG + Knowledge Compiler
Stored source metadata	Sometimes	Not explicit at inference	Present in object fields; verified only for a recovered subset
Typed output	❌	❌	✅ Concept / Threshold / Procedure / ...
Machine-readable	❌	❌	✅ numerical_value, steps[], signs[]
Composable	❌	❌	✅ Object → Agent → Workflow
Structural checks	Varies	Varies	Five automated syntax/shape/relationship checks
Update cost	Low	High	Low (re-run one section)
Agent integration	Probabilistic	Opaque	Deterministic

Knowledge Compiler is a complement to RAG, not a replacement. The typed objects it produces can be indexed by a vector store and retrieved alongside raw chunks — giving agents both structured facts and full-context passages.

---

Structural Checks and Evidence Boundary

The historical corpus is processed by five deterministic automated checks—no LLM judges or probabilistic scoring in this check stage.

Boundary: These checks enforce parseability, field shape, relationship integrity, graph constraints, and content-shape heuristics. They do not establish source fidelity, factual correctness, expert validation, ground truth, or study admissibility.

📖 See docs/knowledge-ir.md for the formal definition of Knowledge IR.

1. Syntax — YAML parses, required fields exist
2. Schema — type-specific fields complete (Threshold has range, Procedure has steps)
3. Ontology — relationship fields are checked against available targets and an allowlist; warnings may remain
4. Graph — legacy graph checks run, but global identity is not guaranteed (11 duplicate-ID groups remain explicit)
5. Content shape — definitions are non-empty, thresholds have numerical bounds

Current status: structural checks pass under the legacy validator; the provenance audit separately identifies 844 Tier-A records, 207 Tier-B records, zero Tier-C study-admissible records, a 571-record strongest recovered subset, and 1,734 quarantined records.

---

Repository structure

knowledge-compiler/
├── ontology_spec.md          # Language spec for the Knowledge IR
├── schema/                   # Type definitions (8 files)
├── prompts/                  # Pipeline prompts (6 files)
├── scripts/                  # Automation
├── compiler/                 # CLI + task queue + provider abstraction
├── sources/                  # Source documents compiled into IR
│   └── books/                # Textbook sources
│       ├── acsm12/           # 707 IR nodes (12 chapters)
│       └── nsca-cscs/        # 1598 IR nodes (26 chapters)
│   (future: papers/ guidelines/ wiki/)
├── knowledge_compiler/       # Skill API for querying compiled IR
├── docs/                     # Pipeline + schema + ontology + IR docs
└── releases/                 # Release plans

---

Quick start

# Clone
git clone https://github.com/GBX-Max1220/knowledge-compiler.git
cd knowledge-compiler

# See it in action — no API keys, no setup
python demo.py

# Validate the objects
python scripts/validate.py --book acsm12
    python scripts/validate.py --book nsca-cscs

Then query the Skill API:

```python
from knowledge_compiler import Skill

skill = Skill("sources/books/acsm12")

# List all registered concepts
for name in sorted(skill.registry):
    print(f"{name:40s} → {skill.registry[name]}")

Running the full pipeline

The pipeline stages (chunk, extract, normalize) use an AI provider to generate objects from textbook content. Two options:

• Reasonix CLI — default provider. Install separately as reasonix and run python scripts/run_queue.py nsca-cscs
• OpenAI — set OPENAI_API_KEY and pass --provider openai to the compiler

---

Potential Applications

• Retrieval and provenance research — inspect structured records and their source-link status
• Exercise-coaching prototypes — engineering exploration only; the corpus is not approved as expert-equivalent recommendation evidence
• Clinical decision-support research — prototype workflow exploration only; not clinically or scientifically validated
• Educational tutoring research — students explore typed concepts; content accuracy has not been manually reviewed
• Scientific knowledge bases — multi-source, versioned, queryable

---

Roadmap

Version	Coverage	Status
v0.1	ACSM 12th Ed. (Ch1-3 pilot) — schema, ontology, prompts	✅ Released
v0.2	ACSM Full Book (707 objects) + NSCA-CSCS Full Book (1598 objects)	✅ Complete
v1.0	Cross-source merge, multi-backend (Neo4j/RDF/MCP), CI/CD, external review	🔜
v2.0	New source types (clinical guidelines, papers), public API	🔜

Future sources:

• WHO Physical Activity Guidelines
• General medical / HCI handbooks
• Research papers (PubMed extraction pipeline)

---

License

MIT

Citation

@software{guo_knowledge_compiler_2025,
  author = {Guo, Max},
  title = {Knowledge Compiler: From Textbook to Structured Skill},
  year = {2025},
  url = {https://github.com/GBX-Max1220/knowledge-compiler}
}