Skip to content

feat: ontology-gap detection — recommend new entity TYPES from extraction misfits + eval gaps #77

Description

@rajnavakoti

Description

Extraction is currently ontology-constrained: when the LLM hits a concept that doesn't fit, it marks it status: proposed and suggests an existing type, and flags new jargon — but it does not recommend a brand-new entity type. Evals surface content gaps (MISSING/INCOMPLETE) but not type gaps. So enriching the ontology from demand is today a semi-manual loop: a human runs an LLM over the proposed-flags + eval gaps to spot missing types.

This issue: make CB recommend new entity types/relationships natively — analyze the proposed/misfit flags + eval coverage gaps and surface "you likely need a new type X / relationship Y" for human review. Part of the bidirectional write-back vision (agent/pipeline proposes → human curates). Related: #19 (MCP write server), #75.

Why (and why it's downstream, not upstream)

The DDC → ontology-enrichment story (e.g. "DDC surfaced the ADR-compliance link that NORIA-O's top-down ontology missed") works semi-manually today and is sufficient for the experiment. Build this AFTER running the manual loop a few times — the real type-gaps that emerge tell you what the feature should recommend. Demand-driven tooling: let the need define the feature.

Acceptance Criteria

  • Aggregate extraction proposed/misfit signals + eval MISSING/INCOMPLETE gaps
  • LLM analysis recommends candidate new entity types and/or relationship fields (with rationale + example instances)
  • Recommendations surfaced for human review (never auto-applied to the ontology)
  • Output is actionable: proposed type name, layer, directory, example entities
  • Tests

Out of Scope

Metadata

Metadata

Assignees

No one assigned

    Labels

    frameworkScientific frameworks

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions