Custom groups for ESLint plugin

[matejchalk]

User story

As a user of @code-pushup/eslint-plugin, I would like to easily configure my own custom groups of ESLint rules with weights I give them. I would like to configure this using the ESLint rule IDs I'm already familiar with (e.g. @typescript-eslint/naming-convention), instead of the slugified audits which often include an options hash (e.g. typescript-eslint-naming-convention-bff213ed8affa645).

Inspiration

• custom ESLint categories in code-pushup/ng-demo
• [eslint-plugin] Match ESLint rules with options #862

Proposal

The eslintPlugin would accept a new options object defined as follows:

type ESLintPluginOptions = {
  groups?: CustomGroup[]
}

type CustomGroup = {
  slug: string,
  title: string,
  description?: string,
  docsUrl?: string,
  rules: CustomGroupRules
}

type CustomGroupRules = string[] | Record<string, number>

Rules will be referenced by their plugin-scoped ESLint rule ID, just like in a regular ESLint config. The eslintPlugin function will take care of transforming it to a Code PushUp group with matching audits. Audits will be matched regardless of if the rule has additional options or not.

For simplicity, the user would be able to specify an array of rule ID references, which assigns weight: 1 for each audit. But if the user doesn't want a uniform distribution, they could use an object instead which maps rules to their weights.

We can further simplify configuration by supporting * wildcards in the rule ID. This syntax would allow a user to group all rules from a specific ESLint plugin together with one reference (e.g. @angular-eslint/* or rxjs-x/*). A wildcard reference will be resolved as if each rule was specified individually, with the same weight applied to all matching audits.

Example

This user configuration:

await eslintPlugin(['src/**/*.ts', 'src/**/*.html'], { groups: [
  {
    slug: 'modern-angular',
    title: 'Modern Angular',
    rules: {
       '@angular-eslint/template/prefer-control-flow': 3,
       '@angular-eslint/prefer-standalone': 3,
       '@angular-eslint/prefer-on-push-component-change-detection': 2,
       '@angular-eslint/template/prefer-ngsrc': 2,
       '@angular-eslint/component-selector': 1
    }
  },
  {
    slug: 'type-safety',
    title: 'Type safety',
    rules: [
      '@typescript-eslint/no-explicit-any',
      '@angular-eslint/template/no-any',
      '@typescript-eslint/no-unsafe-*'
    ]
  },
] })

would create the following groups:

[
  {
    slug: 'modern-angular',
    title: 'Modern Angular',
    refs: [
      { slug: 'angular-eslint-template-prefer-control-flow', weight: 3 },
      { slug: 'angular-eslint-prefer-standalone', weight: 3 },
      { slug: 'angular-eslint-prefer-on-push-component-change-detection', weight: 2 },
      { slug: 'angular-eslint-prefer-ngsrc', weight: 2 },
      { slug: 'angular-eslint-component-selector-bf80e0a74cfb03f8', weight: 1 },
    ]
  },
  {
    slug: 'type-safety',
    title: 'Type safety',
    refs: [
      { slug: 'typescript-eslint-no-explicit-any', weight: 1 },
      { slug: 'angular-eslint-template-no-any', weight: 1 },
      { slug: 'typescript-eslint-no-unsafe-argument', weight: 1 },
      { slug: 'typescript-eslint-no-unsafe-assignment', weight: 1 },
      { slug: 'typescript-eslint-no-unsafe-call', weight: 1 },
      { slug: 'typescript-eslint-no-unsafe-member-access', weight: 1 },
      { slug: 'typescript-eslint-no-unsafe-return', weight: 1 }
    ]
  },
]

which can then be used to create categories:

export default {
  plugins: [
    await eslintPlugin(
      // ...
    )
  ],
  categories: [
    {
      slug: 'modern-angular',
      title: 'Modern Angular',
      refs: [
        { type: 'group', plugin: 'eslint', slug: 'modern-angular', weight: 1 } 
      ]
    },
    {
      slug: 'type-safety',
      title: 'Type safety',
      refs: [
        { type: 'group', plugin: 'eslint', slug: 'type-safety', weight: 1 }
      ]
    }
  ]
}

Acceptance criteria

• eslintPlugin supports optional groups
• extra groups are added to predefined groups
• ESLint rule ID matches audit slug
• ESLint rules with options are also matched
  • if many audits are from the same rule, the weight is distributed evenly between them
• * in rule ID reference acts as wildcard to match many rule IDs with same prefix and/or suffix
  • same weight is assigned to each different rule matched by wildcard
• rules object assigns explicit weight to each rule
• rules array assigns weight 1 to each rule
• document configuration in package README