Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 112 additions & 0 deletions packages/loopover-miner/docs/contribution-profile.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# ContributionProfile schema — AMS per-repo contribution-eligibility rules

Design spike for **#6795**, part of the AMS per-repo contribution-profile epic (#6793). This documents the
`ContributionProfile` shape AMS uses to represent what it has learned about a repo's contribution-eligibility
rules, before any extraction (#6796), caching (#6797) or `discover` wiring (#6798) is built against it.

**Design/schema only — no extraction logic and no `discover` wiring here.** The importable types live in
`packages/loopover-miner/lib/contribution-profile.d.ts`; the constants and two pure helpers in
`contribution-profile.js`.

The shape is **grounded in the real-repo signal inventory** (#6794,
`ams-contribution-signal-inventory.md`), not designed in the abstract. Three of that inventory's findings
directly drove decisions the abstract shape would have gotten wrong — those are called out inline below.

## The core idea: a profile is a bundle of independently-confident signal rules

Every rule in the profile is a `ContributionSignalRule<T>`:

```ts
{ value: T | null; confidence: "explicit" | "inferred" | "absent" | "unknown"; provenance: [...] }
```

- `value` is `null` whenever `confidence` is `absent`/`unknown`, so a consumer can never mistake "no rule" for
"empty rule".
- `confidence` is **per rule, not per repo.** _(Finding #2, #6794: signal quality varies widely_ within _one
repo — `rust` has excellent label descriptions and no PR template; `react` has a PR template and almost no
label descriptions. A single repo-level score would be useless.)_
- `absent` is a first-class value, distinct from `unknown`. _(Finding: 3/10 sampled repos expose no eligibility
label at all — a real answer `discover` must be able to act on, different from "we failed to look".)_
- `provenance` records which signal each rule came from, for debuggability. _(Finding: the primary source
differs per repo — some state rules only in agent docs, some only in labels.)_

## Fields

| Field | Type | Notes |
| ------------------- | -------------------------------------------- | --------------------------------------------------------------------- |
| `repoFullName` | `string` | |
| `schemaVersion` | `number` | Bumped on any shape change, so an older cached profile is detectable. |
| `generatedAt` | `string` | ISO timestamp. |
| `eligibilityLabels` | `SignalRule<ContributionLabelMatcher[]>` | OR-list of matchers; `absent` when the repo has no eligibility label. |
| `exclusionLabels` | `SignalRule<ContributionLabelMatcher[]>` | Usually `inferred` or `absent` (see below). |
| `prBody` | `SignalRule<ContributionPrBodyRequirements>` | Optional slot, not a spine field (see below). |
| `completeness` | confidence | The **weakest** spine signal — see the rule below. |

### Eligibility labels are matchers over name AND description

`ContributionLabelMatcher` is `{ field: "name" | "description"; contains: string }` — a case-insensitive
substring test, not a fixed name list and not a regex (kept auditable).

> **Finding #1 (#6794):** `good first issue` / `help wanted` is present in only 6/10 sampled repos. `rust`,
> `deno` and `kubernetes` — the highest-activity repos — use their own taxonomies, and `rust` encodes the
> "good first issue" meaning **only** in the description of `E-easy` (`"…Good first issue."`), which a
> name-only match misses. So the matcher must be able to test the description, and the rule is an OR-list so a
> repo with several eligibility labels is represented fully.

### Exclusion labels are weaker by construction

> **Finding (#6794):** nothing in the sample marks issues maintainer-only via a label whose _name_ says so;
> the closest signals are status labels (`blocked`, `on-hold`) whose exclusion meaning is conventional, not
> stated. So `exclusionLabels` will usually be `inferred` or `absent`, and the confidence field is what lets
> `discover` weight it accordingly rather than trusting it like an explicit eligibility rule.

### PR-body requirements are an optional slot, not a spine field

> **Finding (#6794):** the linked-issue requirement — the rule loopover's own gate enforces hardest — is
> **loopover-local**, absent from the rest of the sample (loopover 8 mentions in `CONTRIBUTING.md`, react/rust/
> kubernetes 0). Modelling it as a core field would encode our own norm as an ecosystem norm. It lives in the
> optional `prBody` rule, which is `absent` for most repos.

### `completeness` is the weakest signal, not the average

`completeness = weakestConfidence([eligibilityLabels.confidence, exclusionLabels.confidence, prBody.confidence])`.
Weakest wins so one strong signal never masks an absent one — a profile with a crisp eligibility rule but no
exclusion data is still only as trustworthy as its weakest part, and `discover` should treat it conservatively.

## Assignee exclusion is a runtime check, not a stored rule

> **Finding (#6794):** "not assigned to the repo owner" is not documented for most repos and is derivable from
> the issue's own `assignees` field at query time. So it is **not** a profile field. `ContributionAssigneeRuntimeCheck`
> (`{ excludeAssignedLogins: string[] }`) names the live filter #6796/#6798 apply at discover time, keeping a
> runtime concern out of the cached profile.

## Agent docs rank at least as highly as `CONTRIBUTING.md`

Not a schema field, but a note for the extractor (#6796), from the inventory:

> **Finding (#6794):** `AGENTS.md`/`CLAUDE.md` is the most consistently-present non-label signal (6/10), and
> `sure-aio` has `AGENTS.md` and **no** `CONTRIBUTING.md`. Extraction that treats human docs as primary and
> agent docs as fallback has the priority backwards for that repo. `provenance.source` includes `agent_docs`
> as a first-class source for exactly this reason. Also: `CONTRIBUTING.md` lives at the repo root in 6/10 and
> under `.github/` in 2/10, so the extractor must probe both, and treat a very small file (react's is 208 B) as
> a signpost rather than as rules.

## Caching shape

The profile is cached in a local SQLite store keyed by repo, mirroring the miner's other local stores
(`policy-doc-cache.js`), because labels and docs both change over time.

- Table: `CONTRIBUTION_PROFILE_STORE_TABLE` = `"miner_contribution_profile"`.
- TTL: `CONTRIBUTION_PROFILE_CACHE_TTL_MS` = 7 days. Labels/docs change slowly; a week bounds staleness without
re-fetching every run. `CachedContributionProfile.stale` is `true` once `fetchedAt` is older than the TTL,
and the caller re-extracts.
- The store itself is #6797's deliverable; this issue only fixes the table name and TTL so that issue and this
schema agree.

## What the implementation issues build on this

- **#6796 (extraction):** populates each `SignalRule` from labels + `CONTRIBUTING.md` (root and `.github/`) +
PR template + agent docs, setting `confidence`/`provenance` per the findings above.
- **#6797 (cache + doctor):** the `miner_contribution_profile` SQLite store with the TTL above.
- **#6798 (`discover` wiring):** reads the profile's eligibility/exclusion rules and the runtime assignee check
to filter candidate issues, weighting each by its `confidence`.
122 changes: 122 additions & 0 deletions packages/loopover-miner/lib/contribution-profile.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
// ContributionProfile schema (#6795) — the shape AMS uses to represent what it has learned about a repo's
// contribution-eligibility rules, before any extraction (#6796) or `discover` wiring (#6798) is built against
// it. Grounded in the real-repo signal inventory (#6794, packages/loopover-miner/docs/ams-contribution-signal-
// inventory.md), whose findings drove three schema decisions the abstract shape would have gotten wrong:
// 1. Eligibility labels are matchers over name AND description, not a fixed name list — rust/deno/kubernetes
// use their own taxonomies and encode the meaning in the description.
// 2. Every rule is INDEPENDENTLY absent: "absent" is a first-class confidence, distinct from "not yet
// extracted", because signal quality varies widely WITHIN a single repo.
// 3. The linked-issue requirement is NOT a core field — it is loopover-local, absent from the rest of the
// sample — so it lives in an optional `prBody` slot rather than the profile's spine.

/** How trustworthy a single extracted rule is. `explicit`: derived from an unambiguous, machine-readable
* signal (a label whose name/description states eligibility, a CONTRIBUTING line that names a required label).
* `inferred`: derived from a conventional-but-unstated signal (a `blocked` status label read as exclusionary).
* `absent`: the repo exposes no signal of this kind at all — a real, common answer (3/10 of the #6794 sample
* had no eligibility label), and deliberately distinct from `unknown`. */
export type ContributionSignalConfidence =
"explicit" | "inferred" | "absent" | "unknown";

/** Where a rule was derived from, for debuggability (#6794 found the primary source differs per repo — some
* state rules only in agent docs, some only in labels). */
export type ContributionSignalSource =
"labels" | "contributing_md" | "pr_template" | "agent_docs";

export interface ContributionSignalProvenance {
source: ContributionSignalSource;
/** Human-readable pointer to the exact signal, e.g. a label name or a doc path. Never secrets. */
detail: string;
}

/** A matcher for an eligibility/exclusion label. Matches over the label's NAME or DESCRIPTION — #6794 found
* rust encodes "good first issue" semantics only in `E-easy`'s description, which a name-only match misses. */
export interface ContributionLabelMatcher {
/** Which field the pattern tests. */
field: "name" | "description";
/** Case-insensitive substring the field must contain (not a regex — kept simple and auditable). */
contains: string;
}

/** One extracted rule: its value, how confident the extractor was, and what it was derived from. `value` is
* `null` when `confidence` is `absent`/`unknown`, so a consumer never mistakes "no rule" for "empty rule". */
export interface ContributionSignalRule<T> {
value: T | null;
confidence: ContributionSignalConfidence;
provenance: ContributionSignalProvenance[];
}

/** Optional PR-body requirements. Modelled as an optional slot rather than a spine field precisely because
* #6794 found the linked-issue requirement is loopover-local, not an ecosystem norm. */
export interface ContributionPrBodyRequirements {
/** Does a PR need to reference an issue with a closing keyword (Closes/Fixes #N)? */
requiresLinkedIssue: boolean;
}

/** The learned contribution-eligibility profile for one repo. */
export interface ContributionProfile {
repoFullName: string;
/** Bumped when the field set/semantics change, so a cached profile from an older extractor is detectable. */
schemaVersion: number;
/** ISO timestamp the profile was built. */
generatedAt: string;
/** Which label(s) mark an issue contributor-workable. `value` is an OR-list of matchers; `absent` when the
* repo exposes no eligibility label (a real outcome for 3/10 of the #6794 sample). */
eligibilityLabels: ContributionSignalRule<ContributionLabelMatcher[]>;
/** Which label(s) mark an issue maintainer-only / off-limits. Weaker/more inferential than eligibility per
* #6794 (nothing in the sample named exclusion in a label NAME), hence usually `inferred` or `absent`. */
exclusionLabels: ContributionSignalRule<ContributionLabelMatcher[]>;
/** Optional PR-body requirements (see the type). Absent for most repos. */
prBody: ContributionSignalRule<ContributionPrBodyRequirements>;
/** Overall completeness: the least-confident spine signal, so `discover` can treat a partial profile
* conservatively. NOT an average — one strong signal must not mask an absent one. */
completeness: ContributionSignalConfidence;
}

/** Assignee-exclusion (e.g. "not assigned to the repo owner") is deliberately NOT a profile field: #6794 found
* it is not documented for most repos and is derivable from the issue's own `assignees` at query time. This
* type names that runtime check so the implementation issues (#6796/#6798) treat it as a live filter, not a
* cached rule. */
export interface ContributionAssigneeRuntimeCheck {
/** Exclude issues assigned to any of these logins (typically the repo owner). Applied at discover time. */
excludeAssignedLogins: string[];
}

/** A cached profile plus the metadata that governs when it is refreshed. Mirrors the miner's other local
* SQLite stores (policy-doc-cache.js): keyed by repo, with a TTL, because labels and docs both change. */
export interface CachedContributionProfile {
profile: ContributionProfile;
/** ISO timestamp the profile was written to the cache. */
fetchedAt: string;
/** True once `fetchedAt` is older than the store's TTL — the caller should re-extract. */
stale: boolean;
}

export const CONTRIBUTION_PROFILE_SCHEMA_VERSION: 1;
export const CONTRIBUTION_SIGNAL_CONFIDENCE_LEVELS: readonly [
"explicit",
"inferred",
"absent",
"unknown",
];
export const CONTRIBUTION_SIGNAL_SOURCES: readonly [
"labels",
"contributing_md",
"pr_template",
"agent_docs",
];
/** Default cache TTL: 7 days. Labels/docs change slowly; a week bounds staleness without re-fetching per run. */
export const CONTRIBUTION_PROFILE_CACHE_TTL_MS: number;
/** The local SQLite store table the cache (#6797) will use, named here so the schema owns it. */
export const CONTRIBUTION_PROFILE_STORE_TABLE: "miner_contribution_profile";

/** Build an empty, fully-`absent`/`unknown` profile for a repo — the safe default before extraction has run,
* so `discover` treats an unprofiled repo conservatively rather than as "no restrictions". */
export function emptyContributionProfile(
repoFullName: string,
generatedAt: string,
): ContributionProfile;

/** The least-confident of a set of signal confidences, per the `completeness` rule (weakest wins). */
export function weakestConfidence(
confidences: readonly ContributionSignalConfidence[],
): ContributionSignalConfidence;
74 changes: 74 additions & 0 deletions packages/loopover-miner/lib/contribution-profile.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
// ContributionProfile schema constants + tiny pure helpers (#6795). Design/schema only — no extraction logic
// (#6796) and no `discover` wiring (#6798) live here. The shapes are documented in contribution-profile.d.ts
// and packages/loopover-miner/docs/contribution-profile.md; this file exists so the implementation issues have
// concrete, importable constants and the two branch-free helpers they will build on.

/** Bumped when the field set/semantics change, so a cached profile from an older extractor is detectable. */
export const CONTRIBUTION_PROFILE_SCHEMA_VERSION = 1;

/** Confidence vocabulary, weakest-last order used by weakestConfidence. `absent` (the repo has no such signal)
* is deliberately distinct from `unknown` (we have not looked / could not tell). */
export const CONTRIBUTION_SIGNAL_CONFIDENCE_LEVELS = Object.freeze([
"explicit",
"inferred",
"absent",
"unknown",
]);

/** The signal sources a rule can be derived from (#6794 found the primary source differs per repo). */
export const CONTRIBUTION_SIGNAL_SOURCES = Object.freeze([
"labels",
"contributing_md",
"pr_template",
"agent_docs",
]);

/** Default cache TTL: 7 days. Labels/docs change slowly; a week bounds staleness without re-fetching per run. */
export const CONTRIBUTION_PROFILE_CACHE_TTL_MS = 7 * 24 * 60 * 60 * 1000;

/** The local SQLite store table the cache (#6797) will use, named here so the schema owns it. */
export const CONTRIBUTION_PROFILE_STORE_TABLE = "miner_contribution_profile";

/** An `absent` signal rule with no value and no provenance — the safe default for a spine field. */
function absentRule() {
return { value: null, confidence: "absent", provenance: [] };
}

/**
* Build an empty, fully-`absent` profile for a repo — the safe default before extraction has run, so `discover`
* treats an unprofiled repo conservatively rather than as "no restrictions".
*
* @param {string} repoFullName
* @param {string} generatedAt ISO timestamp (passed in rather than read from the clock, so callers/tests stay
* deterministic — mirrors how the other miner builders take their timestamp).
* @returns {import("./contribution-profile.js").ContributionProfile}
*/
export function emptyContributionProfile(repoFullName, generatedAt) {
return {
repoFullName,
schemaVersion: CONTRIBUTION_PROFILE_SCHEMA_VERSION,
generatedAt,
eligibilityLabels: absentRule(),
exclusionLabels: absentRule(),
prBody: absentRule(),
completeness: "absent",
};
}

/**
* The least-confident of a set of signal confidences — the rule behind a profile's `completeness`. Weakest
* wins, so one strong signal never masks an absent one. An empty set is `unknown` (nothing observed).
*
* @param {readonly string[]} confidences
* @returns {string}
*/
export function weakestConfidence(confidences) {
let weakestIndex = -1;
for (const confidence of confidences) {
const index = CONTRIBUTION_SIGNAL_CONFIDENCE_LEVELS.indexOf(confidence);
if (index > weakestIndex) weakestIndex = index;
}
return weakestIndex === -1
? "unknown"
: CONTRIBUTION_SIGNAL_CONFIDENCE_LEVELS[weakestIndex];
}
Loading