Skip to content

Reflection mode: a class-level [MethodDataSource] on a generic test class fails the whole assembly scan #7208

Closed

Description

@thomhurst

What happens

In reflection mode, a class-level [MethodDataSource] on a generic test class aborts discovery of the whole assembly. Every test in the assembly is replaced by a single failing [ASSEMBLY SCAN FAILED] <assembly> entry. A run with a --treenode-filter that does not match that entry reports "Zero tests ran".

[MethodDataSource(nameof(Strings))]
public class OnlyMds<T>(T value)
{
    public static IEnumerable<string> Strings() => ["s"];

    [Test]
    [Arguments("a")]
    public void Test(T arg) { }
}

Reproduced on main (c4b9977) by adding this class to tests/TUnit.TestProject. With --reflection --list-tests the project lists 49 tests instead of thousands. Running --treenode-filter "/*/*/*/AssemblyScanFailed" --reflection shows:

InvalidOperationException: Failed to get data from source: Method, property, or field 'Strings' not found in class 'Object' with the specified arguments.
  at TUnit.Core.MethodDataSourceAttribute.GetDataRowsAsync(...)  MethodDataSourceAttribute.cs:261
  at ReflectionTestDataCollector.GetDataFromSourceAsync(...)      ReflectionTestDataCollector.cs:1419
  at ReflectionTestDataCollector.GroupClassRowsByTypeArgumentsAsync(...) :1077
  at ReflectionTestDataCollector.DiscoverGenericTests(Type)       :753
  at ReflectionTestDataCollector.DiscoverTestsInAssembly(Assembly) :456

GroupClassRowsByTypeArgumentsAsync evaluates the class data sources with metadata whose class is object, so the member lookup fails. The exception is not caught per class, so it escapes to ProcessAssemblyAsync.

Source-gen mode runs the test, but it builds the instance from the method row's value (ctor=a) and ignores the class [MethodDataSource] row ("s").

Expected

  • Reflection resolves the class-level [MethodDataSource] member on the test class, and groups its rows by type as it does for other class sources.
  • A failure while discovering one class is reported as a failed test for that class only, not for the whole assembly.
  • Both modes build the instance from the class source's row.

Related difference (class-row instantiations)

Moved to #7335: source-gen runs only one of two class-level sources that infer the same instantiation. Source-gen also does not infer class type arguments from a class-level [MethodDataSource] at all (#7333), and tuple rows from a class-level [MethodDataSource] do not instantiate a class with several type parameters (#7334).

Found while handling a review thread on #7149.

Activity

  1. added
    P1High: significant user-facing bug, fix soon
    on Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High: significant user-facing bug, fix soonbugSomething isn't workingin-progressWork is in progress

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions