feat(estimator): sub-cluster estimator + PlanExecutor fan-out (PR-2) - #27
Closed
flg77 wants to merge 1 commit into
Closed
feat(estimator): sub-cluster estimator + PlanExecutor fan-out (PR-2)#27flg77 wants to merge 1 commit into
flg77 wants to merge 1 commit into
Conversation
…-out (PR-2)
New acc/estimator.py:
* TaskComplexity dataclass — narrow input surface (tokens, task_type,
required_skills, has_external_io).
* Estimator Protocol — pure callable, easy to fuzz + custom-implement.
* default_estimator: token-budget heuristic (base + ceil(tokens/per_n))
with keyword-driven difficulty bumps. Output clamped to
[1, min(cap, role.max_parallel_tasks)] — defence in depth on top of
Cat-A A-019.
* build_estimator() dispatcher: 'heuristic' (default) | 'fixed' |
'module:dotted.path'. Unknown / failing strategies log + fall back
to heuristic — arbiter NEVER crashes on a buggy operator config.
* slice_skill_mix() round-robins skills across N sub-agents so no one
member loads every skill prompt.
* derive_complexity() turns a raw step payload into TaskComplexity
(token estimate via len(text)//4, [SKILL: ...] hint extraction).
acc/config.py:
* RoleDefinitionConfig gains max_parallel_tasks: int = 1 (legacy: no
parallelisation) and estimator: dict[str, Any] = {}. Schema is a
free-form dict on purpose so 'module:' strategies don't need
schema bumps.
acc/plan.py:
* PlanExecutor.__init__ accepts optional role_resolver / skill_resolver
callbacks. Without them, dispatch is byte-identical to PR-1.
* _maybe_build_cluster: consults the estimator, returns ClusterPlan or
None for single-agent fallback. All exceptions logged + downgraded
to single-agent dispatch.
* _dispatch_cluster: fans one PLAN step out as N TASK_ASSIGN payloads
sharing one cluster_id (PR-1 wire shape).
* on_task_complete: cluster aggregation — step transitions only after
all members report. COMPLETE if every member ok, FAILED if any one
blocked. Cluster auto-unregistered on transition.
regulatory_layer/category_a/constitutional_rhoai.rego:
* Bumped 0.4.0 → 0.5.0.
* Two new rules:
- deny_cluster_oversize: subagent_count > role.max_parallel_tasks
- deny_cluster_nonpositive: subagent_count < 1
Both gate action='CLUSTER_SPAWN' so external (Gatekeeper) admission
enforces the same invariant as the in-process clamp.
26 new tests in tests/test_estimator.py — heuristic shape, role-cap
clamp, [0,1] difficulty bound, skill_mix precedence, fixed strategy,
module: import + import-failure fallback, unknown-strategy fallback,
slice_skill_mix round-robin, derive_complexity SKILL-hint extraction,
PlanExecutor single-agent fallback, fan-out wire-shape, A-019 in-process
clamp, estimator-failure → single-agent fallback, cluster aggregation
all-members-must-report, any-blocked-fails-step.
47 passed across PR-1 + PR-2 test modules; 104 across all related
modules.
Foundation complete for PR-4 (TUI cluster panel) which subscribes via
the PR-1 register_cluster_listener and PR-3 (markdown role authoring,
independent of this PR).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 6, 2026
flg77
added a commit
that referenced
this pull request
May 7, 2026
… reopened) (#31) * feat(cluster): cluster_id propagation foundation (PR-1 of subagent clustering) New acc/cluster.py module: * ClusterPlan dataclass with field invariants (subagent_count >= 1, difficulty in [0, 1]). * In-memory registry (register/lookup/unregister/list) with optional Redis mirror via redis_compat — edge-friendly: works without Redis. * new_cluster_id() emits c-prefixed UUIDs to discriminate from task_id (plan-…) and agent_id (<role>-<hex>) in log lines / dashboards. * fetch_cluster_async() backfills from Redis on local-cache miss. Wire-protocol propagation: * acc/plan.py:_publish_task_assign accepts optional cluster_id + target_agent_id kwargs; both attached only when supplied so legacy single-agent payloads stay byte-identical. * acc/agent.py:_handle_task echoes inbound cluster_id on every outbound TASK_PROGRESS and TASK_COMPLETE so cluster fan-in aggregators see a complete event stream per cluster. TUI fan-out: * NATSObserver gains register_cluster_listener / unregister_cluster_listener + internal _fan_out_cluster helper. Every cluster-tagged TASK_PROGRESS / TASK_COMPLETE fans out to per-cluster_id callbacks with per-callback exception isolation (one buggy listener cannot starve others). Payloads without cluster_id are silently ignored. 21 new tests in tests/test_cluster_propagation.py covering: dataclass invariants, registry round-trip, sync/async lookup miss behaviour, TASK_ASSIGN cluster_id presence/absence, listener fan-out, unregister idempotency, multi-listener support, exception isolation. Foundation for PR-2 (estimator + sub-cluster spawn) and PR-4 (TUI cluster panel). No user-visible change in this PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(estimator): role-driven sub-cluster estimator + PlanExecutor fan-out (PR-2) New acc/estimator.py: * TaskComplexity dataclass — narrow input surface (tokens, task_type, required_skills, has_external_io). * Estimator Protocol — pure callable, easy to fuzz + custom-implement. * default_estimator: token-budget heuristic (base + ceil(tokens/per_n)) with keyword-driven difficulty bumps. Output clamped to [1, min(cap, role.max_parallel_tasks)] — defence in depth on top of Cat-A A-019. * build_estimator() dispatcher: 'heuristic' (default) | 'fixed' | 'module:dotted.path'. Unknown / failing strategies log + fall back to heuristic — arbiter NEVER crashes on a buggy operator config. * slice_skill_mix() round-robins skills across N sub-agents so no one member loads every skill prompt. * derive_complexity() turns a raw step payload into TaskComplexity (token estimate via len(text)//4, [SKILL: ...] hint extraction). acc/config.py: * RoleDefinitionConfig gains max_parallel_tasks: int = 1 (legacy: no parallelisation) and estimator: dict[str, Any] = {}. Schema is a free-form dict on purpose so 'module:' strategies don't need schema bumps. acc/plan.py: * PlanExecutor.__init__ accepts optional role_resolver / skill_resolver callbacks. Without them, dispatch is byte-identical to PR-1. * _maybe_build_cluster: consults the estimator, returns ClusterPlan or None for single-agent fallback. All exceptions logged + downgraded to single-agent dispatch. * _dispatch_cluster: fans one PLAN step out as N TASK_ASSIGN payloads sharing one cluster_id (PR-1 wire shape). * on_task_complete: cluster aggregation — step transitions only after all members report. COMPLETE if every member ok, FAILED if any one blocked. Cluster auto-unregistered on transition. regulatory_layer/category_a/constitutional_rhoai.rego: * Bumped 0.4.0 → 0.5.0. * Two new rules: - deny_cluster_oversize: subagent_count > role.max_parallel_tasks - deny_cluster_nonpositive: subagent_count < 1 Both gate action='CLUSTER_SPAWN' so external (Gatekeeper) admission enforces the same invariant as the in-process clamp. 26 new tests in tests/test_estimator.py — heuristic shape, role-cap clamp, [0,1] difficulty bound, skill_mix precedence, fixed strategy, module: import + import-failure fallback, unknown-strategy fallback, slice_skill_mix round-robin, derive_complexity SKILL-hint extraction, PlanExecutor single-agent fallback, fan-out wire-shape, A-019 in-process clamp, estimator-failure → single-agent fallback, cluster aggregation all-members-must-report, any-blocked-fails-step. 47 passed across PR-1 + PR-2 test modules; 104 across all related modules. Foundation complete for PR-4 (TUI cluster panel) which subscribes via the PR-1 register_cluster_listener and PR-3 (markdown role authoring, independent of this PR). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
flg77
added a commit
that referenced
this pull request
Jun 5, 2026
…surface)
Eighth and final sub-slice of Stage 1
(openspec/changes/20260605-acc-pkg-trust-and-assistant/). Lands the
declarative DC install API surface — the GitOps seam for Stage
1.5.3's pkg-install code path.
What ships:
* operator/api/v1alpha1/acccatalog_types.go (NEW):
- AccCatalog CRD mirroring acc.pkg.catalog.Catalog Pydantic
model so the operator's rendered YAML validates cleanly
against the Python loader.
- Spec fields: catalogId, tier (trusted|tp|community|self),
mode (https|file), url|path, requiredSigner (issuer +
subjectPattern + optional keyPath), priority.
- Status fields: observedGeneration, conditions[],
lastRenderedAt.
- Printcolumns + shortNames for kubectl-friendly UX.
* operator/api/v1alpha1/accpackageinstall_types.go (NEW):
- AccPackageInstall CRD — one @scope/name@constraint install.
- Spec fields: name (regex-validated), constraint, catalogRef
(optional pin), targetCorpus (optional scope), allowUnsigned
(operator-explicit bypass).
- Status fields: phase (Pending|Installing|Installed|Failed),
installedVersion, installPath, contentSha256, lastInstalledAt,
conditions[].
* operator/api/v1alpha1/zz_generated_stage1_6_deepcopy.go (NEW):
- Hand-written DeepCopy / DeepCopyObject methods following
controller-gen's emission style. Replace on next
`make generate`.
* gitops/argocd/applications/accpackage-sample.yaml (NEW):
- End-to-end ArgoCD Application driving two AccCatalog entries
(canonical https + corp-internal file-mode) + two
AccPackageInstall objects. Operators copy + adjust the
catalog URL + signer pattern for their environment.
* gitops/argocd/applications/README.md (NEW):
- Documents what Stage 1.6 ships (API + sample) vs what's
deferred to 1.6b (reconcilers + RBAC + OLM bundle +
envtest). The deferred reconciler consumes the same
fetch_and_install Python entry point that 1.5.3 and 1.4
already use — single seam, no parallel logic.
Design choices:
* API surface lands now so downstream GitOps tooling can import
the types; reconciler logic (exec-into-pod, leader election,
status patching) is multi-day Go work and ships as 1.6b.
* AccCatalog Spec mirrors acc.pkg.catalog.Catalog 1:1 — operator
renders directly to /etc/acc/catalogs.yaml ConfigMap, no
translation layer.
* AccPackageInstall.spec.constraint accepts the same range syntax
acc.pkg._semver implements; operator does shape validation, the
installer is the resolution authority.
* AllowUnsigned at the CR level so dev-environment opt-out is
declarative + audit-logged via the controller's events.
Tests: no new Python tests (Go-only changes; Python pkg suite
unchanged at 415/1 green). Go envtest integration tests land in
1.6b alongside the reconciler. Manifest sample YAML parses (5
documents: Application + 2 AccCatalog + 2 AccPackageInstall).
This completes Stage 1's eight sub-slices:
1.5.1 dual-source role loader (#21)
1.5.2 required_packages (#22)
1.5.3 acc-deploy.sh boot-time fetch (#23)
1.4 PROPOSE_INFUSE marker (#24)
1.1 eval format (#25)
1.2 EC policy depth (#26)
1.3 OIDC keyless publish (#27)
1.6 operator CRDs (this)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
flg77
added a commit
that referenced
this pull request
Jun 5, 2026
…I variant End-to-end runner that walks the operator through the five-phase acc1 K8s hub smoke after PRs #20-#28 land. Hermetic CI variant exercises the same chain in-process against a file-mode catalog so PR-time tests prove the wiring without a running cluster. What ships: * tools/smoke-acc1-hub.sh (NEW): - Phase 0 preflight — checks cosign / kubectl / python / jq / curl / acc-pkg on PATH. - Phase 1 — applies gitops/acc-hub/ if not present; waits for rollout; curls /index.json. - Phase 2 — generates pilot cosign keypair via tools/cosign-pilot-keygen.sh if not on disk. - Phase 3 — builds pilot pkg, signs with cosign sign-blob, publishes via gitops/acc-hub/publish-to-hub.sh; verifies the hub now advertises the package via jq on the live index. - Phase 4 — downloads tarball + sig from live hub, runs acc-pkg install into a tmp sandbox, exercises cosign verify. - Phase 5 — RoleLoader resolves coding_agent from the installed-package path (proves the dual-source loader chain from PRs #21-#23). - Coloured logging + idempotent steps + smoke-specific exit codes (7 = hub deploy fail, 8 = roundtrip verification fail). * tests/pkg/test_live_smoke_hermetic.py (NEW): - Mirrors the bash script's Phase 3-5 in-process against a file-mode catalog with mocked cosign so CI exercises the chain without acc1 reachability. - 7 tests: build determinism, end-to-end install + load, idempotent re-install, signing-floor refusal, --allow-unsigned bypass, PROPOSE_INFUSE shares the same fetch_and_install seam, and smoke script wiring sanity (script references the right helpers). * tools/SMOKE.md (NEW): - Operator runbook: prerequisites, run command, what each phase does, exit codes, troubleshooting matrix. Test growth: 2979/37 (PR #27 baseline) -> 422/1 pkg suite (this PR adds +7 hermetic tests on top of the operator-only script). Full sweep impact is +7 (since #28 was Go-only, no Python tests). Stage 1 close-out — every code path the eight sub-slices ship is now exercised by a single hermetic test that proves they compose correctly: Build (#20) -> Sign (#27) -> Publish (#27) -> Catalog resolve (#20) -> Verify (#20 + #26) -> Install (#20) -> Registry (#20) -> RoleLoader (#21) -> PROPOSE_INFUSE dispatch (#24) all hit the same code path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TaskComplexity+EstimatorProtocol +default_estimator(heuristic) +build_estimatorstrategy dispatcher (heuristic | fixed | module:dotted.path) +slice_skill_mix+derive_complexity.RoleDefinitionConfiggainsmax_parallel_tasks: int = 1andestimator: dict[str, Any] = {}(free-form dict so module: strategies don't need schema bumps).role_resolver/skill_resolverconstructor kwargs; when wired, the executor consults the role's estimator and may fan one PLAN step out as N TASK_ASSIGN payloads sharing one cluster_id. Aggregation inon_task_completewaits for all members. Without resolvers, dispatch is byte-identical to PR-1.deny_cluster_oversize,deny_cluster_nonpositive).tests/test_estimator.py; 47 green across PR-1 + PR-2; 104 across all related modules.Why
Step 2 of
docs/PLAN_subagent_clustering.md. Lets roles declare how they parallelise; A-019 + the in-process clamp are defence-in-depth so a buggy custom estimator can never spawn beyondrole.max_parallel_tasks.Stacking
This PR targets
feat/cluster-id-propagation(PR #26). Merge order: PR #26 → this PR.Test plan
pytest tests/test_estimator.py tests/test_cluster_propagation.py— 47 passed🤖 Generated with Claude Code