Skip to content

Mutants that do not compile count as KILLED, which inflates the score (Go: + on strings) #18

Open

Description

@szybnev

What happens

In Go, + → - is applied to string concatenation, as in role + " " + name or "sha256:" + hex. The mutant does not compile:

invalid operation: operator - not defined on role (variable of type string)
FAIL	example.com/demo/skills [build failed]

The README says that "a replacement that does not compile fails the test run and counts as killed". So these sites always count as killed, even though no test was involved.

In a small package with 10 covered mutants, 4 were compile failures of this kind, and the score showed 100% instead of 6/6 killed by tests. In Go code that builds messages, paths or SQL, string + is common, so every package score is inflated a little.

The same rule also hides real problems. When a worker cannot build at all (for example the go:embed case in #15), every mutant is "killed" and the run reports 100%.

Suggestion

  • Report a fourth outcome, such as INVALID or NOT COMPILED, in the table and the EDN snapshot, and leave it out of the score.
  • For Go, the build failure is easy to recognise: [build failed] or [setup failed] in go test output, or an exit code from go vet/go build before the tests run.
  • Optionally, skip + → - in Go when the operand is a string literal. That catches the most common case without type information.

Environment

mutator a5a0cb8, Go 1.27.1.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions