Feature: Honor scoped `applyTo` in `.github/instructions/` via CWD-derived workspace scope

[franklixuefei]

Problem summary

Conductor v0.1.17's --workspace-instructions discovers .github/instructions/**/*.instructions.md but filters them through _is_always_on_instructions_file, which loads ONLY files with applyTo: "**" (config/instructions.py:145-166). Files with any scoped applyTo glob — **/*.cs, tests/**, src/**, services/foo/**, etc. — are silently skipped.

GitHub Copilot's documented applyTo semantic is per-path scoping: a file with applyTo: "**/*.cs" applies whenever any C# file is involved; one with applyTo: "services/foo/**" applies to that service's tree. These are correct uses of the convention, not misuse. Conductor's all-or-nothing filter doesn't honor them — it treats anything other than the literal "**" as "skip unconditionally," which excludes the bulk of real-world per-area instructions.

The gap applies to any repo that uses applyTo for scoping — single-service flat repos with per-language scoping, library repos splitting src/ vs examples/ vs docs/, and monorepos with per-service scoping.

Evidence (ground-truthed against upstream/main, conductor v0.1.17)

Where the filter lives

src/conductor/config/instructions.py:

# L177
CONVENTIONS: list[Convention] = [
    ConventionFile("AGENTS.md"),
    ConventionFile(".github/copilot-instructions.md"),
    ConventionFile("CLAUDE.md"),
    ConventionDirectory(
        path=".github/instructions",
        pattern="*.instructions.md",
        include_file=_is_always_on_instructions_file,   # ← the filter
        recursive=True,
    ),
]

# L145
def _is_always_on_instructions_file(path: Path) -> bool:
    fm = _parse_frontmatter(path)
    if fm is None:
        return False
    return fm.get("applyTo") == "**"   # L166 — exact-equality "**" only

CWD is already available at the right seam: cli/run.py:1223 passes Path.cwd() as auto_discover_dir to discover_workspace_instructions. The plumbing for CWD-implicit scoping exists; only the per-file filter needs to learn how to use it.

Real-world data (Azure Chaos Studio)

Inventory of .github/instructions/*.instructions.md files in a real Azure repo. Most globs are language- or area-scoped rather than service-scoped — i.e., the same pattern that appears in single-service repos:

File	applyTo	Loaded by conductor today?
arm-rpc-guidelines.instructions.md	services/GW/**;services/BE/**	❌
csharp-coding-standards.instructions.md	**/*.cs	❌
eng_ms.instructions.md	/docs/eng.ms/*.*; /docs/eng.ms/**/*.*	❌
engineering-standards.instructions.md	**/*.cs,**/*.csproj,**/Directory.Packages.props,**/global.json,**/*.bicep,**/*.md	❌
poc-warning.instructions.md	(uses non-standard glob: field, no applyTo)	❌
portal-ux-patterns.instructions.md	**/portal-extension/**,**/portal-ux/**,**/ux-accelerator/**	❌
specs-arm-api.instructions.md	docs/specs/arm-api-published/*.*; docs/specs/arm-api-published/**/*.*	❌
testing-standards.instructions.md	**/*Tests.cs,**/*Tests/**,**/*UnitTests*/**,**/*HermeticTests*/**	❌
workload-identity.instructions.md	**/kubectl/*.yaml,**/kubectl/*.yml,**/identity.bicep,**/ev2/**	❌

Total: 52 KB / ~13K tokens. 0 / 9 files loaded today. All 9 contain real engineering guidance (ARM RPC patterns, async/await rules, testing categories, AKS workload identity setup, etc.).

These globs are not misuse of applyTo — they correctly express the convention's per-path scoping. Promoting them all to applyTo: "**" would falsify the convention's intent and inject (e.g.) AKS workload-identity guidance into agents working on portal-extension code.

Repro

mkdir repro && cd repro
git init -q
mkdir -p .github/instructions src tests

echo "Repo-wide rule." > .github/copilot-instructions.md

cat > .github/instructions/csharp.instructions.md <<'EOF'
---
description: C# coding standards
applyTo: "**/*.cs"
---
Use file-scoped namespaces.
EOF

cat > .github/instructions/testing.instructions.md <<'EOF'
---
description: Test-specific patterns
applyTo: "tests/**"
---
Tests must use the Arrange-Act-Assert pattern.
EOF

cat > wf.yaml <<'EOF'
workflow:
  name: repro
  entry_point: agent_a
agents:
  - name: agent_a
    prompt: "Echo your workspace_instructions."
    routes:
      - to: $end
EOF

conductor run --workspace-instructions wf.yaml

Expected: agent prompt's <workspace_instructions> block contains content from csharp.instructions.md and testing.instructions.md.
Actual: block contains only copilot-instructions.md. Both scoped files are silently skipped.

Proposed design

Three pieces. The first is the correctness fix; the second is the minimum generalization to keep the seam clean for future scoped conventions; the third is inspectability so the implicit behavior is debuggable.

Up-front honest framing

At repo-root CWD — which is the dominant case for programmatic launchers (skyship Invoke-Ship.ps1, octane run-phases.py, ATS-Copilot pr-reviewer, all of which cd to the user's repo root before invoking conductor) — bidirectional overlap means "every scoped file overlaps with the root subtree" → load everything. At the root, this design is effectively "drop the applyTo filter." That is strictly better than today's silent skip (correctness fix), but it is not token narrowing for programmatic callers. Narrowing only kicks in when a human cds into a subdir (cd services/GW/ && conductor run …).

This is a deliberate tradeoff: correctness over token economy for v1. Programmatic narrowing can be tackled later if/when token bloat at the repo root becomes a measured problem.

Piece 1 — CWD-implicit scope filter

Replace _is_always_on_instructions_file's applyTo == "**" check with bidirectional glob-overlap against the existing auto_discover_dir (CWD).

Algorithm:

1. From CWD, walk up to find the workspace root (git root, or deepest ancestor with a known convention — same logic conductor already uses for discovery).
2. Compute CWD-relative-to-workspace-root (e.g., services/GW/, or . when CWD is the workspace root).
3. For each discovered instructions file, ask the convention for the scope glob (see Piece 2).
4. Load the file if any of:
   • The convention has no scope concept (e.g., AGENTS.md) — load unconditionally, matching today's behavior.
   • extract_scope returns None — equivalent to "always on" (covers both applyTo: "**" and unscoped frontmatter).
   • The scope glob's subtree overlaps with the CWD-relative subtree, in either direction.

Bidirectional overlap. applyTo: services/GW/** matches when CWD is services/GW/src/ (user is inside the scope) and when CWD is . (the scope is inside the user's CWD subtree). Strict one-direction glob match would exclude one of these common cases. A "shares-any-literal-prefix-or-could-match-any-file-under-CWD" approximation is sufficient; precise glob intersection is undecidable in general but unnecessary here.

Piece 2 — Minimal generalization of the existing seam

ConventionDirectory.include_file: Callable[[Path], bool] | None (line 77) is the existing pluggable filter slot — exactly the right place to extend. Two minimal changes:

@dataclass(frozen=True)
class ConventionDirectory:
    path: str
    pattern: str
    extract_scope: Callable[[Path], str | None] | None = None  # NEW
    recursive: bool = True
    include_file: Callable[[Path], bool] | None = None  # retained briefly for back-compat

The convention author declares only how to read their scope field. The universal overlap test lives in core (one source of truth):

# .github/instructions/ registration
ConventionDirectory(
    path=".github/instructions",
    pattern="*.instructions.md",
    extract_scope=_extract_apply_to,
)

def _extract_apply_to(path: Path) -> str | None:
    fm = _parse_frontmatter(path)
    if fm is None:
        return None  # no frontmatter → no scope claim → load
    apply_to = fm.get("applyTo")
    if apply_to is None or apply_to == "**":
        return None  # "always on" semantic → load
    return apply_to

_walk_directory_convention does the overlap test against auto_discover_dir once, generically:

if convention.extract_scope is not None:
    scope = convention.extract_scope(file_path)
    if scope is not None and not _scope_overlaps(scope, cwd_rel_to_root):
        continue

When a second scoped convention shows up (e.g., Cursor's .cursor/rules/*.mdc with globs: frontmatter), it registers its own extract_scope — no new abstraction, no core changes, same overlap semantic.

Not introducing: a ConventionHandler Protocol / registry. With exactly one scoped convention today, the right shape for a registry is undervalidated; extending the existing ConventionDirectory slot is the YAGNI-honoring minimum. Revisit if a third scoped convention reveals a shape the dataclass can't accommodate.

Piece 3 — --print-loaded-instructions (v1 deliverable)

Implicit behavior trades discoverability for ergonomics — same command in different directories loads different files. Ship inspectability alongside the implicit filter, not after.

conductor run --workspace-instructions --print-loaded-instructions wf.yaml
# prints the resolved list of loaded instruction files (with their effective
# scope and the overlap reason) to stderr before launching the workflow

Could also fold into existing --verbose output instead of a new flag if that's preferred — open to either. The requirement is that "why isn't my instruction loading" is answerable without instrumenting source code.

What we're explicitly NOT doing

• No new --workspace-scope flag. The repeatable --instructions <path> flag (cli/app.py:326-329, instructions.py:410-424) already exists as the force-load escape hatch for callers that know exactly what they want. Adding a third knob would be one too many. Programmatic launchers that need finer narrowing than CWD can list specific files via --instructions, or cd first.
• No ConventionHandler Protocol / registry. YAGNI — extend the existing ConventionDirectory slot until a second scoped convention validates the registry shape.

Principles satisfied

1. Convention over configuration. CWD is the universal CLI primitive every tool (git, grep, npm) already uses. No new concept for users.
2. Single source of truth. CWD is where the user "is"; no parallel flag to keep in sync.
3. Faithful to the convention's own semantics. applyTo is path-glob scoping; evaluating against actual filesystem paths is what the convention describes. The overlap test lives in core (one place), not duplicated across convention authors.
4. Right architectural seam. The fix lives in conductor's discovery layer (config/instructions.py), upstream of every provider adapter. One change, all providers and all callers benefit.
5. Graceful degradation. Conventions without a extract_scope keep working unchanged. New conventions plug in with one callable.
6. Symmetric primitive. Same mechanism (CWD) handles both ends of the scope spectrum: cd repo-root = broad, cd services/GW/src/ = narrow.
7. Composable with existing tools. Anyone wrapping conductor in a script can narrow scope by cd-ing first. Standard Unix composition.
8. YAGNI. No --workspace-scope flag, no ConventionHandler registry — both can land later if real cases demand them.
9. Backward compatibility. extract_scope is a new optional field; include_file retained during migration. No existing convention author is broken.

Honest tensions

1. Implicit behavior trades discoverability for ergonomics. Addressed by Piece 3 (--print-loaded-instructions) shipping in the same change, not as a follow-up.
2. At repo-root CWD, this loads all scoped files. Stated plainly above. Acceptable v1 tradeoff (correctness > token economy); programmatic narrowing can come later as --workspace-scope or similar if measured to matter.
3. extract_scope -> str | None implicitly commits to "scope = path glob." Today's two known scoped conventions (applyTo, Cursor's globs:) both fit. A wildly different scope semantic (RBAC role, language ID) would need a different mechanism — acceptable per YAGNI.
4. Glob-overlap semantics need care. The bidirectional-overlap predicate is more nuanced than strict glob match. Mitigation: an approximation (shared literal prefix + wildcard tolerance) is sufficient; precise intersection is unnecessary and undecidable in general.

Comparison with the existing --instructions <path> flag

Axis	--instructions <path> (today)	This proposal (CWD-implicit + extract_scope)
What you declare	Specific files to load	Nothing (CWD is implicit)
Who decides relevance	Caller (must enumerate files)	Conductor (consults each convention's scope predicate)
Convention semantics	Bypassed — files loaded unconditionally	Honored — applyTo / globs / etc. respected via convention's extract_scope
Per-call cardinality	N flags (one per file)	Zero — CWD-implicit
Caller knowledge required	File inventory + per-file relevance	Just cd to the right place

Complementary, not redundant. --instructions remains as the force-load escape hatch ("ignore the convention, give me these files"). This proposal is the principled mechanism ("honor the convention; here's where I'm working").

Integration impact: most callers need zero changes

Programmatic launchers across the Microsoft scenarios all already cd to the user's repo root before invoking conductor:

Caller	How it invokes conductor today	Behavior after fix
Skyship Invoke-Ship.ps1 (stack / fix-review)	conductor run from repo root	✅ All applyTo-matching files load automatically (= correctness fix; not narrowing)
Octane pr-orchestrator run-phases.py	conductor run from repo root	✅ Same
Octane octane-workflow-implement	conductor run from repo root	✅ Same
ATS-Copilot pr-reviewer	conductor run from repo root	✅ Same
Manual conductor run from SKILL.md docs	User's CWD	✅ Honors wherever the user is
User cd services/GW && conductor run …	services/GW/	✅ Narrows to GW automatically — no flag needed

No follow-up integration PRs needed in skyship / octane / ATS for the correctness fix. The --workspace-instructions adoption PRs already shipped (or pending review) are exactly the right pre-work; the conductor fix lights them up.

Affected users

Anyone using --workspace-instructions against a repo whose .github/instructions/ files use scoped applyTo globs. Affected categories include single-service flat repos (**/*.cs, tests/**), library repos (src/ vs examples/ vs docs/), framework-scoped repos (**/*.tsx, scripts/**), and monorepos with per-service scoping (the most acute case).

Direct evidence: Azure Chaos Studio (9 files, 0 loaded today; 6/9 globs are language- or area-scoped and would appear identically in non-monorepo repos). Copilot Chat's docs explicitly recommend applyTo for per-area scoping in any repo, so this pattern is conventional, not anomalous.

[franklixuefei]

Hi Jason Robert (@jrob5756) , this is the design for filling the gap of instructions loading for conductor. LMK if it looks good and I can go ahead with the implementation. Thanks!

[brrusino]

Thanks for the thorough writeup, Frank Li (@franklixuefei) — the diagnosis is dead-on. _is_always_on_instructions_file literally compares applyTo == "**" and drops everything else, so any scoped file gets silently skipped. That's a real bug, not a misuse of the convention. The Azure Chaos table makes it concrete.

A few reactions:

Layer 1 — yes, this is the right direction. We already pass Path.cwd() as auto_discover_dir from the CLI, so the CWD signal is sitting right there. Deriving scope from it is the natural move and doesn't ask anything new of callers. I don't want to introduce any new configuration here — CWD is where the user already is, let's just use it.

Layer 3 — I'd skip it entirely. --instructions <path> already exists as the force-load escape hatch for callers that know exactly what they want. Adding --workspace-scope on top gives us three knobs (--workspace-instructions + --instructions + --workspace-scope) where one-and-a-half would do. If a programmatic launcher really cares about narrowing, it can use --instructions with the file list it already has. I don't think we need a third mechanism.

Layer 2 — I'd defer it. We have exactly one convention with scope semantics today (applyTo). ConventionDirectory.include_file is already the right seam to extend — either let the predicate take a CWD-relative path, or swap it for an extract_scope() + generic overlap test. A full ConventionHandler registry is the right idea once a second scoped convention shows up; until then it's abstraction we'd be paying for without a second concrete case to validate the shape against.

One thing I want us to be honest about up front. When CWD is the repo root (which is the dominant case for skyship / octane / pr-orchestrator), bidirectional overlap means "scope is inside my CWD subtree" → load everything scoped. So at the root, Layer 1 is effectively "drop the applyTo filter." That's strictly better than today (which silently skips), but it's not narrowing in the common programmatic case — the narrowing only kicks in when a human actually cds into a subdir. I think that's the right tradeoff (correctness beats token economy here), but I'd rather we say it plainly in the design than imply it magically narrows for everyone.

One small ask: ship a way to inspect what got loaded — either a --print-loaded-instructions flag or folding it into existing verbose output. The implicit behavior is fine, but the first time someone debugs "why isn't my instruction loading," they're going to want this.

So my vote: green-light Layer 1, skip Layer 3, defer Layer 2. Happy to talk more if you want to push back on any of this.

[franklixuefei]

Thanks Brent Rusinow (@brrusino) — code-grounded against main (config/instructions.py, cli/run.py:1223), all three of your points hold:

• The filter is literally applyTo == "**" at line 166.
• Path.cwd() is already the auto_discover_dir at cli/run.py:1223 — no new plumbing needed for Layer 1.
• ConventionDirectory.include_file (line 77, called only at :241 and :262) is the single filter seam — exactly the right place to extend.

Layer 1 — accepted. Going with CWD-implicit, no new flag.

Layer 3 — accepted, dropped. You're right that --instructions + cd cover the narrowing cases that actually matter. Three knobs would be one too many.

Layer 2 — accepted, no registry; one preference. Of your two sub-options, I'd vote for extract_scope() + generic overlap test in core, not parameterized include_file(path, cwd_rel). Reason: the overlap semantic (what does "scope overlaps CWD" mean) is universal — if it lives in each convention's predicate, the second scoped convention either duplicates the helper or risks subtly different semantics. Keeping the overlap test in core and asking convention authors only for "here's how I read my scope field" keeps the abstraction minimal and keeps the semantic in one place. Concretely:

@dataclass(frozen=True)
class ConventionDirectory:
    path: str
    pattern: str
    extract_scope: Callable[[Path], str | None] | None = None  # new, optional
    recursive: bool = True
    # include_file retained briefly for backward compat; removed once migrated.
    include_file: Callable[[Path], bool] | None = None

# .github/instructions/ registration becomes:
ConventionDirectory(
    path=".github/instructions",
    pattern="*.instructions.md",
    extract_scope=_extract_apply_to,   # returns applyTo string, or None for "**"/missing
)

_walk_directory_convention then does the universal overlap test against auto_discover_dir once. When/if a second scoped convention shows up (Cursor's globs:, etc.), it just registers its own extract_scope — no Protocol, no registry, no core changes.

Honest framing — agreed, will state plainly up front. Revised design will lead with: "At repo-root CWD (the dominant case for programmatic launchers), bidirectional overlap loads every scoped file — effectively dropping the applyTo filter at the root. That's strictly better than today's silent skip but is not narrowing for the programmatic case; narrowing only kicks in when a human cds into a subdir. Correctness > token economy is the deliberate v1 tradeoff."

--print-loaded-instructions — committed to v1. Will land as part of the same change, not a separate follow-up. Foot-gun avoidance, as you say.

Going to update the issue body to reflect this (drop the registry framing, drop Layer 3, fold in the honest framing and the inspectability commitment), then start on the implementation unless you flag anything in the revised body. Thanks again for the sharp read — design is meaningfully simpler and more honest because of it.