Skip to content

Latest commit

 

History

History
345 lines (256 loc) · 14.7 KB

query.md

File metadata and controls

345 lines (256 loc) · 14.7 KB
estado Completed

Query Engine

Rootline exposes data through a declarative query model.

Queries return records, not rendered documents. Derived and aggregated fields are computed automatically and appear alongside frontmatter in results.

CLI Usage

The scan target is the positional path, if supplied, otherwise --from (default .). The governance-boundary preflight checks that same target, not the working directory. For example, rootline query docs/ --from other/ scans and checks docs/; a governed working directory cannot authorize an undeclared target, and an undeclared working directory does not block a target with its own declared boundary.

The --where flag accepts expr-lang/expr expressions. Multiple --where flags are combined with AND:

rootline query --where 'status == "published"'
rootline query --where 'tipo in ["lxc", "vm"]' --where 'estado not in ["Completed"]'

Sorting

The --sort flag accepts a comma-separated list of field:direction pairs. Direction defaults to asc if omitted:

rootline query --sort "prioridad:asc,impact_score:desc"
rootline query --where 'estado == "Pending"' --sort "prioridad:asc"
rootline query docs/ --sort "path:desc"

The scalar builtins path, body, and type sort directly from the record, before any same-named derived or frontmatter field. Other fields resolve derived values before frontmatter. Sorting by path:asc or path:desc orders record paths deterministically in ascending or descending order.

Sort type detection per field:

  1. Enum fields: If the field has a .stem schema with type: enum and values: [...], sorts by index position in the values list.
  2. Numeric fields: If both values parse as float64, sorts numerically.
  3. String fields: Fallback to lexicographic comparison.

Missing keys and nil values (including explicit YAML null) always sort last, regardless of direction. Null/missing peers retain input order when all sort keys are equal; a derived nil value shadows any frontmatter value. Sort applies after filtering and before limit.

Unlike --where, an unknown --sort field is an error, matching the treatment a bad direction already got:

$ rootline query docs/ --sort "estadoo:asc"
Error: unknown sort field "estadoo": no record carries it and no .stem in scope declares it (did you mean "estado"?)

The known names are the same set --where checks against, taken from the corpus before --where narrows it, so a filter that matches nothing does not invalidate a valid sort key. The complex builtin sections is not sortable and always fails explicitly, even when there are no known fields:

$ rootline query docs/ --sort "sections"
Error: unsupported sort field "sections": complex builtin fields cannot be sorted

When there are neither records nor schema fields to check against, unknown names remain accepted; the explicit sections rejection still applies.

Link Traversal Predicates

Query documents based on their link relationships using --has-inbound and --has-outbound:

# Find records with inbound links from documents matching a filter
rootline query --has-inbound 'tipo == "epic"'

# Find records with outbound links matching a filter (empty = any)
rootline query --has-outbound ''

# Restrict to specific link types
rootline query --has-inbound 'tipo == "epic"' --inbound-type 'depends'
rootline query --has-outbound 'estado == "published"' --outbound-type 'blocks'

# Combine with other filters
rootline query --where 'estado == "Pending"' --has-inbound 'true'

Link-traversal predicates apply --where filters to linked records and keep records with at least one matching link. When combined with --where, both filters AND together. Predicates support both [[wiki-links]] and [markdown](links) reference styles.

Flags:

  • --has-inbound "<expr>" — Keep records with inbound links from records matching expr (empty = any record)
  • --has-outbound "<expr>" — Keep records with outbound links to records matching expr (empty = any record)
  • --inbound-type <type> — Restrict --has-inbound to links of this type (e.g., depends, blocks)
  • --outbound-type <type> — Restrict --has-outbound to links of this type
  • --graph-root <path> — Set the root for edge scanning (default: the query path). The query path must lie inside the graph root.

Link styles: By default, link traversal searches wiki-link ([[target]]) styles only. Configure .stem links.styles as a replacement set, for example [wikilink, markdown], to include markdown links ([text](target)).

Compact Projections with --select

The --select flag produces compact output containing only specified fields:

rootline query --select path,estado
rootline query --select path,estado,links --output jsonl
rootline query --where 'estado == "Pending"' --select path,estado --output csv

--select accepts a comma-separated list of field names. Supported field names include:

  • path — document file path
  • Any frontmatter field name (e.g., estado, prioridad, owner)
  • Any derived field (defined in .stem derive: section)
  • links — document links extracted from wiki-link references

Without --select, query results include full records with frontmatter, body, sections/body_sections, and derived; links is present only when the record has extracted links.

With --select, only specified fields are included in each row, reducing noise and making output suitable for shell pipelines.

Output Formats

The --output flag supports multiple formats:

Format Encoding Requires --select Use case
json (default) Full JSON objects in a rows array No AI agents, typed languages, complete data
table Human-readable columnar format No Terminal inspection, reports
jsonl One JSON object per line Yes Streaming processors, log ingestion
csv RFC 4180 CSV with headers Yes Spreadsheets, SQL import, awk/cut pipelines

Constraint: jsonl and csv formats require --select to specify which columns to output. --count supports only json and table and cannot be combined with --select. Invalid combinations are rejected before schema discovery or scanning.

query is the only command that implements all four formats; every other command declares json and table and rejects the rest. An unknown value such as -o sdlkfj is rejected everywhere. See Output Formats.

Examples:

# JSON (default): full records in a rows array
rootline query --select path,estado --output json
# {"version": 1, "kind": "rootline/query", "rows": [{"path": "...", "estado": "..."}, ...]}

# JSONL: one record per line, no outer wrapper
rootline query --select path,estado --output jsonl
# {"path": "...", "estado": "Pending"}
# {"path": "...", "estado": "In Progress"}

# CSV: RFC 4180 with quoted fields and escaping
rootline query --select path,estado --output csv
# path,estado
# "docs/api/auth.md","Pending"
# "docs/api/endpoints.md","In Progress"

# CSV with nil values (rendered as empty)
rootline query --select path,estado,owner --output csv
# path,estado,owner
# "docs/api/auth.md","Pending","alice"
# "docs/api/endpoints.md","In Progress",""

Column order in jsonl and csv output follows the order specified in --select.

CLI Field Names

Field Type Description
<field> (e.g., estado, impact_score) any Bare frontmatter, derived, or aggregate field name; derived values override same-named frontmatter
path string Record path
type string Record type
body string Full document body text
sections map[string]string Map of heading → section body (requires AST extraction)

Use bare metadata names consistently in --where, --sort, and --select:

rootline query docs/ --where 'estado == "Pending" && impact_score > 0' --sort "impact_score:desc" --select path,estado,impact_score

The CLI expression environment promotes frontmatter and derived keys to top-level names; it does not expose frontmatter or derived namespace maps. Do not use frontmatter.estado or derived.impact_score as namespace paths in these flags. The scalar builtins path, body, and type are available for filtering and sorting; sections is filterable but not sortable. Projection supports path, links, and bare metadata names as described above.

This input contract is separate from --field, which extracts paths from the JSON output envelope: --field 'rows[].frontmatter.estado' is valid for full records without --select.

Section Queries

When AST extraction is enabled, sections exposes a map of heading text to section body content:

rootline query --where 'sections["## Summary"] contains "migration"'
rootline query --where 'sections["## Changelog"] != nil'
rootline query --where 'sections["## References"] contains "[[T001]]"'

The heading key must match the heading text exactly, including the # prefix and any leading/trailing spaces.

Universal Filtering: The --where flag is not limited to query. It is also available on tree, stats, graph, and validate --all. All transversal commands share the same expr-lang syntax.

Field Warnings: Unknown field names in --where expressions emit warnings to stderr with fuzzy suggestions (e.g., warning: unknown field "estdo" in where expression (did you mean "estado"?)). Queries still execute — warnings are informational only.

The warning fires on every command that accepts --where: query, stats, tree, graph and validate --all. Without it a misspelled field is indistinguishable from "no records match" — zero results, empty stderr, exit 0 — which reads as a green check in CI.

In an expression type always names the record field, never a function: the expression language ships a type() builtin of its own, which shadowed the field and made the documented name unusable, so it is disabled. type(x) is therefore not callable in a filter. No other builtin is affected.

A field name is considered known when it is a query builtin (path, body, type, sections), a key any record in the scanned corpus carries, or a field the effective .stem chain declares — including derive: and aggregate: names. The union is deliberately generous: a false warning on a name that does work would teach callers to ignore the warning. On a corpus with neither records nor schema there is nothing to check against, and nothing is reported.

It stays a warning, not an error. A field absent from every record is a legal filter that yields zero matches, and pipelines depend on that exit code.

Source-Backed Fields

A query resolves a canonical body source through the same effective schema as validation:

notes:
  type: string
  source: body.section["## Notes"]

A frontmatter override wins over the extracted value. An empty-present section resolves to ""; duplicate matching headings fail rather than returning one occurrence. Inherited source bindings stay stable, so a child omission inherits the parent binding and a changed binding is a schema conflict.

Operators

Standard expr-lang operators apply:

Operator Semantics Example
== Equals status == "published"
!= Not equals status != "draft"
in One of tipo in ["lxc", "vm"]
not in Not one of estado not in ["draft", "obsoleto"]
contains Substring match body contains "migration"
!= nil Field exists tags != nil
&& AND tipo == "lxc" && estado == "Pending"
|| OR estado == "Pending" || estado == "Bloqueada"

Lower-Level Query API (Condition Trees)

The internal query.Execute API accepts Go Condition trees, not CLI --where expressions or a CLI JSON input option. Its field resolver accepts explicit frontmatter.<field> paths; bare metadata names are expanded to those paths. It does not resolve derived.<field> paths.

result, err := query.Execute(records, &query.Query{
    Where: &query.Condition{
        Op: query.OpAnd,
        Children: []query.Condition{
            {Op: query.OpEq, Field: "frontmatter.status", Value: "published"},
            {Op: query.OpExists, Field: "frontmatter.owner"},
        },
    },
    Limit: 50,
})

Condition trees use declarative operators (eq, ne, in, contains, exists, and). For CLI filtering, use bare names instead, for example --where 'status == "published" && owner != nil'. Do not copy the condition field paths above into CLI expressions.

The separate query.ExecuteExpr API uses the same bare-name expression environment as CLI filtering, not prefixed namespace maps.

Query Result Shape

{
  "version": 1,
  "kind": "rootline/query",
  "meta": {
    "count": 1
  },
  "rows": [
    {
      "path": "docs/api/endpoints.md",
      "type": "markdown",
      "frontmatter": {
        "title": "Endpoints",
        "status": "published"
      },
      "derived": {
        "slug": "endpoints"
      }
    }
  ]
}

Rows include derived when the effective .stem defines derive: or aggregate: fields.

Count Result Shape

rootline query <path> --count supports json (default) and table only. JSON preserves the versioned count envelope:

{
  "version": 1,
  "kind": "rootline/count",
  "meta": { "count": 12 },
  "count": 12
}

With --output table, the same count is one decimal row under a Count column:

$ rootline query docs/ --count --output table
Count
-
12

AI Agent Compatibility

Some AI coding agents (notably Claude Code's Bash tool) pre-escape the ! character to \! in command strings before shell processing. This breaks != expressions at the expr-lang parser level with error unrecognized character U+005C '\'.

Affected operators: !=, != nil, and any expression containing !.

Workarounds (in order of preference):

  1. not in operator — Direct replacement for != negation:

    # Instead of:
    rootline query --where 'estado != "draft"'
    # Use:
    rootline query --where 'estado not in ["draft"]'
  2. in with positive list — Enumerate accepted values:

    rootline query --where 'estado in ["pendiente", "parcial"]'
  3. JSON + jq pipeline — Query all, filter externally:

    rootline query -o json | jq '.rows[] | select(.estado != "draft")'

Field existence check (!= nil): use field not in [nil] when an agent cannot send != nil safely:

rootline query --where 'tags not in [nil]'

This issue is not specific to rootline — it affects any CLI tool whose query syntax uses ! when invoked through agents with this escaping behavior.