Skip to content

Uninferable method data sources share a discovery-failure test ID #7595

Open

Description

@thomhurst

Summary

Multiple method data sources that cannot infer a generic test class's types produce discovery failures with the same test ID. Each failure should have its own identity so test runners can report every source independently.

Reproduction

On the generic-class method-source inference implementation at df2cf32:

public class Tests<T>
{
    public static IEnumerable<object[]> First() => [[1]];
    public static IEnumerable<object[]> Second() => [[2]];
    [Test, MethodDataSource(nameof(First)), MethodDataSource(nameof(Second))]
    public void Run(T value) { }
}

Use a targeted class filter with reflection discovery. BuildTestsFromMethodRows creates separate failure metadata for First and Second. DiscoveryFailureDataSource reaches the per-metadata failure fallback with a default TestDataCombination. GenerateFailedTestId includes class/method names and default combination indexes, but neither source identity nor the exception message, so both IDs are equal. This is established by the code path; an IDE streaming crash has not been reproduced locally.

Expected and actual

Expected: two separately identifiable discovery failures. Actual: both failure metadata entries derive the same test ID. Consolidate the per-method uninferred messages into one violation metadata entry with separately indexed data sources, or otherwise preserve source identity in the failure IDs. Retain constraint failures and sources whose declared inference returns null.

Environment

.NET SDK 10.0.403, net10.0. The same GenerateFailedTestId construction and per-metadata fallback exist on current main 69d3960; the new generic-class inference route is not on main yet. Triage against origin/main e6c7dee confirms that BuildTestsFromMethodRows still handles only ArgumentsAttribute; closed, unmerged #7571 is now replaced by #7596 and #7597. Related open/closed identity issues were searched; #7554 concerns an undiagnosed Windows session hang and is not evidence of this cause.

Reported as a nonblocking review finding while validating #7571: #7571 (comment). Please preserve a distinct failed-test identity for each source.

Acceptance criteria

Prerequisite: #7597, which introduces the affected discovery route. This follow-up is separate from parent #7224's discovery-parity acceptance criteria.

Activity

  1. added
    bugSomething isn't working
    P2Medium: edge case, mode divergence, flaky test, or workaround exists
    on Nov 4, 2026
  2. thomhurst commented on Nov 4, 2026

    @thomhurst
    OwnerAuthor

    Triage: replaced the broad #7224 dependency with native blocked_by #7597, the replacement discovery implementation after #7571 closed unmerged. Added failure-identity acceptance criteria; removed the redundant blocked label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium: edge case, mode divergence, flaky test, or workaround existsbugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions