Skip to content

Add threat-model artifacts and dynamic /goal discovery to the default topology #182

Closed

Description

@aviggiano

Summary

Add threat modeling and parallel /goal vulnerability discovery to the default Ultrafuzz topology. The workflow produces durable threat-model artifacts, generates relevant threat and vulnerability-class goals, expands those goals through first-class dynamic nodes, and preserves producer node IDs on findings.

This is a default-on v0.1.0 workflow change.

Dependencies

The threat-model workflow must consume a validated immutable checkout of aviggiano/web3-vulnerability-database; do not add or duplicate packages/vulnerability-db in Ultrafuzz.

Motivation and references

Effective vulnerability discovery requires an explicit threat model and many focused, parallel goals. A fixed number of generic strategies limits coverage when a protocol has tens or hundreds of distinct threats. A vulnerability taxonomy should improve coverage without triggering irrelevant hunts such as liquidation analysis against a protocol with no liquidation functionality.

Default topology

existing setup/recon
        |\\
actors/roles/flows       existing Ultrafuzz discovery
        |
        v
threat-model ---> fixed roaming /goal
        v
goal-plan <--- validated pinned external database snapshot
        |\\
        | threat-goals (one per threat)
        ` class-goals (one per applicable class)

all discovery lanes -> candidate aggregation -> dedupe/validation -> report bundle

threat-model begins as soon as foundational actor/flow evidence is available. Existing analysis continues in parallel. Generated lanes run under the existing workflow concurrency limits.

threat-model node

The node is named exactly threat-model and produces:

  • THREAT_MODEL.md for humans;
  • canonical threat-model.json for routing and provenance.

Adapt scope/repository evidence, assets and value stores, actors/roles/privileges, trust boundaries, entry points/attack surfaces, threats/preconditions/impact, coverage gaps, assumptions, and unknowns from the referenced skill. Add Web3-specific protocol archetypes, value/accounting flows, economic/security invariants, lifecycle transitions, collateral/debt/solvency/shares/liquidation where present, oracle dependencies, callbacks/hooks/reentrancy, governance/timelocks/upgrades, and cross-chain concerns when applicable. ARI/LINDDUN/privacy analysis and a remediation backlog are out of scope for the main hunting path.

Threat and capability IDs are stable hierarchical IDs such as liquidation:overdue, oracle:stale-price, and accounting:bad-debt. Capabilities use present, absent, or unknown; unknown is never equivalent to absent. Applicability decisions cite their evidence. Generate Markdown from validated JSON so the two artifacts cannot silently diverge.

goal-plan node and external database

Read threat-model.json and the validated pinned external database checkout. Produce schema-validated goal-plan.json containing stable goal/node IDs, full prompt replacements, selected class/threat context, applicability decisions, source digests, database repository/commit/schema/aggregate provenance, and exact selected-record metadata.

The database is fetched/extracted through the same secure immutable-reference cache/materialization behavior used for properties. Keep the validated checkout available during planning; report bundles contain only vulnerability-db-manifest.json and exact selected Markdown records.

The policy is additive, not Cartesian and not top-eight:

one goal per threat
+ one goal per applicable vulnerability class
+ one fixed roaming goal

Unknown capability state remains visible and does not suppress a hunt. A threat model with 100 threats may create 100 threat nodes.

Dynamic prompts

Use #179 to create one child per threat and one child per applicable class. IDs remain meaningful, for example:

dynamic.threat.liquidation.overdue
dynamic.class.liquidation.fixed-term-before-overdue

Prompts must preserve ordinary MDX/template replacement semantics and provide full context:

Your /goal is to find any vulnerability affecting overdue liquidation using threat model threat {{liquidation:overdue}}.
Your /goal is to find a vulnerability of type {{class:liquidation:fixed-term-before-overdue}} using threat model {{liquidation:overdue}}.

Do not flatten placeholders into bare literal IDs before rendering.

Fixed roaming goal

Always run one static /goal node that challenges missing assets/value flows, incorrect trust boundaries, omitted attack surfaces, false assumptions, uncatalogued bug classes, and economic invariants outside selected classes.

Provenance and reporting

  • Runtime assigns producer_node_id to artifacts/candidates.
  • goal-plan.json maps each generated node ID to threat/class context.
  • Aggregation/dedupe preserves and unions source_nodes.
  • The final report renders source node IDs.

Detailed threat content stays in THREAT_MODEL.md and threat-model.json. report.md changes only for concise coverage/artifact links and finding provenance.

Required report-bundle additions are THREAT_MODEL.md, threat-model.json, goal-plan.json, vulnerability-db-manifest.json, and exact selected class records.

Tests and validation

Do not add fake-agent tests. Deterministic checks cover schemas, IDs, present/absent/unknown, additive cardinality, zero applicable classes plus roaming, 100-threat plans, MDX replacements, external snapshot/digest binding, provenance through dedupe, report inclusion, and default-topology concurrency integration. Real-agent validation belongs to #183 and uses only the three existing CI ultrafuzz-bench targets.

Acceptance criteria

  • threat-model is enabled in the default topology.
  • Every run produces validated THREAT_MODEL.md and threat-model.json.
  • goal-plan.json emits one threat goal per threat and one class goal per applicable external-database class.
  • One fixed roaming /goal always runs.
  • Dynamic prompts use full MDX/template replacements.
  • No fixed eight-hunter or other small hard-coded goal limit remains.
  • Existing concurrency limits govern all generated nodes.
  • Findings preserve generated source node IDs through dedupe/reporting.
  • Report bundles contain threat, plan, and selected external-database artifacts.
  • report.md remains concise.
  • Deterministic tests and the real-protocol gate pass without fake agents.

Activity

  1. added this to the v0.1.0 milestone on Aug 4, 2026
  2. aviggiano commented on Aug 4, 2026

    @aviggiano
    CollaboratorAuthor

    Tracked by the v0.1.0 meta plan in #185. Treat the contracts and dependency ordering there as part of this implementation handoff.

  3. aviggiano commented on Aug 7, 2026

    @aviggiano
    CollaboratorAuthor

    Shipped on release/v0.1.0 via PR #188 (db92714). Not on main — see #185 for why, and note GitHub does not auto-close on a merge to a non-default branch.

    .ultrafuzz/topology.yml gains threat-model, goal-plan, threat-goals and class-goals. The earlier concern about a four-model lane firing unattended was refuted during triage: there is no cron in .github/workflows, and push runs the smoke lane only, so the full lane is manual-dispatch.

    Tracked in #327.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions