Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion packages/loopover-miner/DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,11 +66,12 @@ For provider selection and the CLI-specific model/timeout overrides, see
policy-verdict-cache.sqlite3 # cache of resolved AI-usage-policy verdicts (#4843)
deny-hook-synthesis.sqlite3 # synthesized PreToolUse deny-hook proposals (#4522)
orb-export.sqlite3 # opt-in anonymized Orb telemetry export state (#4277)
contribution-profile-cache.sqlite3 # cached per-repo AMS contribution-eligibility signals (#6797)
```

Not every file appears immediately: `laptop-state` is written by `init`, and each of the others is created
the first time its subsystem actually runs (an attempt, a discovery pass, a replay, an Orb export, …), so a
fresh install that has only run `status`/`doctor` will show a subset. All sixteen default into this one
fresh install that has only run `status`/`doctor` will show a subset. All seventeen default into this one
directory. Override the directory for every store at once with `LOOPOVER_MINER_CONFIG_DIR` or
`XDG_CONFIG_HOME` (same resolution chain as `@loopover/mcp`); every store except `laptop-state.sqlite3`
(directory only) also honors its own `LOOPOVER_MINER_<NAME>_DB` path override — e.g.
Expand Down
10 changes: 10 additions & 0 deletions packages/loopover-miner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ tenant goal spec to the ranker, printing `usedDefaultGoalSpec` so a fall-back to
rather than silent. See [`docs/repo-agnostic-capability-audit.md`](docs/repo-agnostic-capability-audit.md) for the
#4780 audit this executes.

`discover` also filters candidates against each target repo's own contribution-eligibility conventions before
ranking/enqueueing (#6793/#6798): it resolves a `ContributionProfile` per repo — generic label-taxonomy/
CONTRIBUTING.md extraction (`lib/contribution-profile-extract.js`), cached locally with a 7-day TTL
(`lib/contribution-profile-cache.js`) — and excludes a candidate missing a required eligibility label, carrying
an exclusion label, or assigned to the repo's own owner login. **Safe-default posture:** a label rule only
excludes when the profile actually resolved a matcher for it; a repo whose conventions couldn't be read at all
(no eligibility/exclusion signal found) excludes nothing via those rules rather than silently skipping real
work. `--json` and the human-readable summary both surface any excluded candidates and why
(`excludedByEligibility`). See [`docs/contribution-profile.md`](docs/contribution-profile.md) for the schema.

The package also includes repo stack auto-detection: `detectRepoStack` (`lib/stack-detection.js`) inspects an
already-cloned target repo's manifest / lockfile / config files and returns a structured description — language,
package manager, and the build / test / lint / format commands — for Node (npm/yarn/pnpm/bun), Python
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import type { ContributionProfile } from "./contribution-profile.js";
import type { RawCandidateIssue } from "./opportunity-fanout.js";

export type CandidateEligibilityResult = {
excluded: boolean;
reasons: string[];
};

export type ExcludedCandidate = {
issue: RawCandidateIssue;
reasons: string[];
};

export type FilterCandidatesResult = {
eligible: RawCandidateIssue[];
excluded: ExcludedCandidate[];
};

export function evaluateCandidateEligibility(
issue: RawCandidateIssue,
profile: ContributionProfile | null | undefined,
options?: {
labelDescriptionsByName?: Map<string, string | null>;
excludeAssignedLogins?: string[];
},
): CandidateEligibilityResult;

export function filterEligibleCandidates(
issues: RawCandidateIssue[],
profilesByRepo: Map<string, ContributionProfile>,
options?: {
labelDescriptionsByRepo?: Map<string, Map<string, string | null>>;
},
): FilterCandidatesResult;

export function profileNeedsLabelDescriptions(profile: ContributionProfile | null | undefined): boolean;

export function fetchRepoLabelDescriptions(
repoFullName: string,
options?: {
fetchImpl?: typeof fetch;
githubToken?: string;
apiBaseUrl?: string;
},
): Promise<Map<string, string | null>>;
174 changes: 174 additions & 0 deletions packages/loopover-miner/lib/contribution-profile-eligibility.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
// Contribution-eligibility filtering (#6798): the last piece of the AMS contribution-profile epic (#6793).
// Applies a target repo's ContributionProfile (#6795/#6796) to `discover`'s fanned-out candidates, excluding ones
// that fail the profile's eligibility rules -- BEFORE ranking/enqueueing, so a real repo's own conventions are
// respected instead of surfacing a candidate its maintainer would reject on arrival.
//
// Safe-default posture: a label-based rule only ever excludes when the profile actually resolved a matcher list
// for it (`value` non-null). A repo with no discoverable eligibility/exclusion label signal at all -- `value:
// null`, the "degrade honestly" outcome #6796 produces for an unreadable or convention-less repo -- excludes
// nothing via that rule. This falls straight out of "there is nothing to check a candidate against," not a
// separate confidence gate bolted on top: `discover` never silently skips real work on a repo whose conventions
// it simply couldn't read. The one check that is NOT gated on profile confidence is assignee-exclusion: per the
// schema (#6795), it is a live, structural fact ("this issue is assigned to the repo owner"), not something
// extraction infers, so it applies to every candidate regardless of how much (or how little) the profile knows.
//
// Conflicting signals (a candidate matches both an eligibility AND an exclusion matcher) resolve conservatively:
// exclusion always wins. The eligibility check only ever ADDS a reason when NO matcher matched; the exclusion
// check only ever ADDS a reason when one DID -- so a candidate that clears eligibility but also trips exclusion
// carries just the exclusion reason, and is excluded either way.

const DEFAULT_API_BASE_URL = "https://api.github.com";
const GITHUB_API_VERSION = "2022-11-28";
const REQUEST_TIMEOUT_MS = 10_000;

function labelMatchesMatcher(matcher, labelNames, labelDescriptionsByName) {
const term = matcher.contains.toLowerCase();
if (matcher.field === "description") {
if (!labelDescriptionsByName) return false;
return labelNames.some((name) => {
const description = labelDescriptionsByName.get(name.toLowerCase());
return typeof description === "string" && description.toLowerCase().includes(term);
});
}
return labelNames.some((name) => name.toLowerCase().includes(term));
}

function anyMatcherMatches(matchers, labelNames, labelDescriptionsByName) {
return matchers.some((matcher) => labelMatchesMatcher(matcher, labelNames, labelDescriptionsByName));
}

/**
* Evaluate one candidate issue against a repo's ContributionProfile. Never throws; a missing/malformed `profile`
* behaves exactly like a fully-absent one (excludes nothing via the label rules).
*
* @param {import("./opportunity-fanout.js").RawCandidateIssue} issue
* @param {import("./contribution-profile.js").ContributionProfile | null | undefined} profile
* @param {{ labelDescriptionsByName?: Map<string, string | null>, excludeAssignedLogins?: string[] }} [options]
* @returns {{ excluded: boolean, reasons: string[] }}
*/
export function evaluateCandidateEligibility(issue, profile, options = {}) {
const reasons = [];

const eligibilityMatchers = profile?.eligibilityLabels?.value;
if (Array.isArray(eligibilityMatchers) && eligibilityMatchers.length > 0) {
if (!anyMatcherMatches(eligibilityMatchers, issue.labels, options.labelDescriptionsByName)) {
reasons.push("missing eligibility label");
}
}

const exclusionMatchers = profile?.exclusionLabels?.value;
if (Array.isArray(exclusionMatchers) && exclusionMatchers.length > 0) {
if (anyMatcherMatches(exclusionMatchers, issue.labels, options.labelDescriptionsByName)) {
reasons.push("exclusion label present");
}
}

// Structural, always-on (not gated on profile confidence, see header comment): defaults to the repo's own
// owner login, already present on the candidate -- no extra lookup needed.
const excludeAssignedLogins = options.excludeAssignedLogins ?? [issue.owner];
const normalizedExcludedLogins = new Set(excludeAssignedLogins.map((login) => String(login).toLowerCase()));
if (issue.assignees.some((login) => normalizedExcludedLogins.has(String(login).toLowerCase()))) {
reasons.push("excluded assignee");
}

return { excluded: reasons.length > 0, reasons };
}

/**
* Split fanned-out candidates into eligible and excluded, per each candidate's own repo profile.
*
* @param {import("./opportunity-fanout.js").RawCandidateIssue[]} issues
* @param {Map<string, import("./contribution-profile.js").ContributionProfile>} profilesByRepo
* @param {{ labelDescriptionsByRepo?: Map<string, Map<string, string | null>> }} [options]
* @returns {{
* eligible: import("./opportunity-fanout.js").RawCandidateIssue[],
* excluded: Array<{ issue: import("./opportunity-fanout.js").RawCandidateIssue, reasons: string[] }>,
* }}
*/
export function filterEligibleCandidates(issues, profilesByRepo, options = {}) {
const eligible = [];
const excluded = [];
for (const issue of issues) {
const profile = profilesByRepo.get(issue.repoFullName) ?? null;
const labelDescriptionsByName = options.labelDescriptionsByRepo?.get(issue.repoFullName);
const result = evaluateCandidateEligibility(issue, profile, { labelDescriptionsByName });
if (result.excluded) {
excluded.push({ issue, reasons: result.reasons });
} else {
eligible.push(issue);
}
}
return { eligible, excluded };
}

/** True when a profile carries at least one description-field matcher -- the only case that needs the repo's
* full label list (name + description), not just the names `discover` already has on each candidate. Keeps the
* extra `/labels` fetch conditional on actually needing it, instead of paying it for every repo. */
export function profileNeedsLabelDescriptions(profile) {
const matcherLists = [profile?.eligibilityLabels?.value, profile?.exclusionLabels?.value];
return matcherLists.some(
(matchers) => Array.isArray(matchers) && matchers.some((matcher) => matcher.field === "description"),
);
}

function githubHeaders(githubToken) {
const headers = {
accept: "application/vnd.github+json",
"user-agent": "loopover-miner",
"x-github-api-version": GITHUB_API_VERSION,
};
if (githubToken) headers.authorization = `Bearer ${githubToken}`;
return headers;
}

function parseRepoFullName(repoFullName) {
if (typeof repoFullName !== "string") return null;
const [owner, repo, extra] = repoFullName.split("/");
if (!owner?.trim() || !repo?.trim() || extra !== undefined) return null;
return { owner: owner.trim(), repo: repo.trim() };
}

/**
* Fetch a repo's full label list (name + description) as a `Map<lowercased name, description>`. Never throws:
* any transport/HTTP/parse failure or malformed repo name yields an empty map, so a fetch problem degrades a
* description-field matcher to "no match" rather than aborting discovery. Only worth calling when
* `profileNeedsLabelDescriptions` is true for the target repo.
*
* @param {string} repoFullName
* @param {{ fetchImpl?: typeof fetch, githubToken?: string, apiBaseUrl?: string }} [options]
* @returns {Promise<Map<string, string | null>>}
*/
export async function fetchRepoLabelDescriptions(repoFullName, options = {}) {
const target = parseRepoFullName(repoFullName);
if (target === null) return new Map();

/* v8 ignore next -- the global-fetch default is the production path; every test injects fetchImpl. */
const fetchImpl = options.fetchImpl ?? fetch;
const base =
typeof options.apiBaseUrl === "string" && options.apiBaseUrl.trim()
? options.apiBaseUrl.replace(/\/+$/, "")
: DEFAULT_API_BASE_URL;
const headers = githubHeaders(options.githubToken);

let response;
try {
response = await fetchImpl(`${base}/repos/${target.owner}/${target.repo}/labels?per_page=100`, {
method: "GET",
headers,
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
});
} catch {
return new Map();
}
if (!response.ok) return new Map();
const payload = await response.json().catch(() => null);
if (!Array.isArray(payload)) return new Map();

const byName = new Map();
for (const label of payload) {
if (typeof label?.name === "string") {
byName.set(label.name.toLowerCase(), typeof label.description === "string" ? label.description : null);
}
}
return byName;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
import type { ContributionProfile } from "./contribution-profile.js";
import type { ContributionProfileCache } from "./contribution-profile-cache.js";

export type ResolveContributionProfilesResult = {
profilesByRepo: Map<string, ContributionProfile>;
labelDescriptionsByRepo: Map<string, Map<string, string | null>>;
};

/** The read/write surface this module actually needs to inject a cache without depending on the SQLite store
* (`dbPath`/`close` are lifecycle concerns the caller, not the resolver, owns) -- mirrors the narrower
* `PolicyDocCache`/`PolicyVerdictCache` convenience types those sibling caches export; contribution-profile-
* cache.js doesn't export an equivalent narrowed type itself, so it's defined locally here instead. */
export type ContributionProfileCacheReader = Pick<ContributionProfileCache, "get" | "put">;

export function resolveContributionProfiles(
repoFullNames: string[],
options?: {
cache?: ContributionProfileCacheReader | null;
fetchImpl?: typeof fetch;
githubToken?: string;
apiBaseUrl?: string;
nowMs?: number;
generatedAt?: string;
},
): Promise<ResolveContributionProfilesResult>;
74 changes: 74 additions & 0 deletions packages/loopover-miner/lib/contribution-profile-resolution.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
import { extractContributionProfile } from "./contribution-profile-extract.js";
import { fetchRepoLabelDescriptions, profileNeedsLabelDescriptions } from "./contribution-profile-eligibility.js";

// Per-repo ContributionProfile resolution for `discover` (#6798): reads a fresh profile from the cache (#6797)
// when one exists and is not stale, otherwise extracts (#6796) live and writes the result back. A caller with no
// cache (dry-run, matching the existing policyDocCache/policyVerdictCache convention of never opening a
// not-yet-existing store during a dry run) always extracts live -- same "read-only GETs are fine, but no local
// store writes" rule discover-cli.js already applies to the other two caches.

/**
* Resolve one repo's profile: cache hit (fresh) wins; otherwise extract live and best-effort write the result
* back to the cache. A cache write failure is non-fatal (the profile itself is still returned) -- the cache is a
* pure performance optimization, never a requirement for discover to have a profile to filter with.
*/
async function resolveOneProfile(repoFullName, options) {
if (options.cache) {
const cached = options.cache.get(repoFullName, options.nowMs);
if (cached && !cached.stale) return cached.profile;
}
const profile = await extractContributionProfile(repoFullName, {
fetchImpl: options.fetchImpl,
githubToken: options.githubToken,
apiBaseUrl: options.apiBaseUrl,
generatedAt: options.generatedAt,
});
if (options.cache) {
try {
options.cache.put(profile, options.nowMs);
} catch {
// Non-fatal: a corrupt/unwritable cache must never block discover from using the profile it just extracted.
}
}
return profile;
}

/**
* Resolve every distinct repo's ContributionProfile (and, only where the profile actually needs it, its full
* label list for description-field matching) in parallel.
*
* @param {string[]} repoFullNames
* @param {{
* cache?: import("./contribution-profile-cache.js").ContributionProfileCache | null,
* fetchImpl?: typeof fetch,
* githubToken?: string,
* apiBaseUrl?: string,
* nowMs?: number,
* generatedAt?: string,
* }} [options]
* @returns {Promise<{
* profilesByRepo: Map<string, import("./contribution-profile.js").ContributionProfile>,
* labelDescriptionsByRepo: Map<string, Map<string, string | null>>,
* }>}
*/
export async function resolveContributionProfiles(repoFullNames, options = {}) {
const profilesByRepo = new Map();
const labelDescriptionsByRepo = new Map();

await Promise.all(
repoFullNames.map(async (repoFullName) => {
const profile = await resolveOneProfile(repoFullName, options);
profilesByRepo.set(repoFullName, profile);
if (profileNeedsLabelDescriptions(profile)) {
const descriptions = await fetchRepoLabelDescriptions(repoFullName, {
fetchImpl: options.fetchImpl,
githubToken: options.githubToken,
apiBaseUrl: options.apiBaseUrl,
});
labelDescriptionsByRepo.set(repoFullName, descriptions);
}
}),
);

return { profilesByRepo, labelDescriptionsByRepo };
}
Loading