Skip to content

Add Agent Playbook section for automated updates #275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stefanvacareanu7 merged 1 commit into from
Jul 29, 2026
Merged

Add Agent Playbook section for automated updates #275

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,65 @@ Example:
</AnalysisInput>
```

## Agent Playbook: Updating This Repository End-to-End

This section is written for an AI coding agent (or a human) tasked with updating this repo — most commonly bumping the wrapped SonarC# (SonarAnalyzer.CSharp) version, but also base image / orb / NuGet dependency bumps. Follow it top to bottom; it tells you what to change, how to regenerate derived files, how to test locally, and how to interpret CI so you can iterate on failures without guessing.

### 1. What this repository is

This is a **Codacy engine**: a .NET (C#, `net6`) wrapper (`src/Analyzer`, built on the `Codacy.Engine.Seed` NuGet package) that packages [SonarC#](https://github.com/SonarSource/sonar-dotnet) (SonarSource's `SonarAnalyzer.CSharp` Roslyn analyzer) as a Docker image Codacy's platform can run against a customer's source code. The build/orchestration layer is a `Makefile` invoked from a multi-stage `Dockerfile` (no Scala/sbt here, and no Python/Node tooling — everything is `dotnet` CLI + `make`).

The `docs/` directory is machine-consumed configuration, just like in the Scala-based Codacy engines:

- `docs/patterns.json` — the full list of SonarC# rules ("patterns") Codacy knows about, their parameters/defaults, and which are enabled by default. Generated file, do not hand-edit.
- `docs/description/description.json` + `docs/description/*.md` — human-readable titles/descriptions per pattern, used in the Codacy UI. Generated file, do not hand-edit.
- `docs/tests/*` and `docs/multiple-tests/*` — fixtures used by `codacy-plugins-test` to validate the engine actually produces the results it claims to for real code samples.
- `docs/tool-description.md` — short blurb about the tool, hand-maintained.

The generator is **`src/DocsGenerator`** (a separate .NET console project, `src/DocsGenerator/Program.cs`). It does **not** clone a GitHub repo like the Scala engines' `DocGenerator` does — instead, `make update-docs` downloads the actual SonarSource release **JAR** (`sonar-csharp-plugin-<version>.jar`) from `https://github.com/SonarSource/sonar-dotnet/releases/download/<version>/sonar-csharp-plugin-<version>.jar`, unzips it, and extracts `Rules.json` plus each rule's `S*.json`/`S*.html` files into `.res/`. `DocsGenerator` then reads those extracted files, plus a `.SONAR_VERSION` marker file, and writes `docs/patterns.json` and `docs/description/*`. This means regenerating docs needs **network access** to GitHub, plus `unzip` and `xmllint` (`libxml2-utils`) installed locally — no `pandoc` needed.

### 2. Files that encode versions — check all of these on every update

| File | What it controls | What to check |
|---|---|---|
| `src/Analyzer/Analyzer.csproj` → `SonarAnalyzer.CSharp` PackageReference `Version` | The wrapped SonarC# analyzer version. This is also read by `Makefile`'s `SONAR_VERSION` var (via `xmllint`) to know which release JAR/tag to fetch for doc regeneration. | Bump to the target version (format `<major>.<minor>.<build>.<revision>`, e.g. `9.32.0.97167`). Confirm a matching GitHub release tag exists at [SonarSource/sonar-dotnet releases](https://github.com/SonarSource/sonar-dotnet/releases). |
| `src/Analyzer/Analyzer.csproj` → `Microsoft.CodeAnalysis.CSharp.Workspaces`, `SQLitePCLRaw.core`, `Codacy.Engine.Seed` | Roslyn workspace API and other NuGet deps | Bump independently if asked (dependabot also opens PRs for these); not tied to SonarC# bumps. |
| `docs/patterns.json` → `"version"` field | The version string Codacy displays for this tool | Regenerated automatically by `DocsGenerator` — do not hand-edit; just verify it matches after regeneration. |
| `.circleci/config.yml` → `codacy/base` orb | Shared CircleCI steps (checkout, versioning, docker build/publish, tagging) | Check the latest published version in the CircleCI orb registry (`git log -p .circleci/config.yml` shows prior bump history as a fallback reference). |
| `.circleci/config.yml` → `codacy/plugins-test` orb | Runs `codacy-plugins-test` in CI after the image is built | Same as above. |
| `Dockerfile` → `ARG DOTNET_VERSION` / `ARG DOTNET_BASE_OS` | .NET SDK/runtime version and Alpine base used to build and run the analyzer | Only bump if the new SonarAnalyzer.CSharp version raises its minimum .NET requirement, or if asked explicitly — don't bump opportunistically. |

### 3. Step-by-step update procedure

1. **Bump `SonarAnalyzer.CSharp` (and any other targeted dependency)** in `src/Analyzer/Analyzer.csproj`, and `.circleci/config.yml` orbs if in scope.
2. **Regenerate the docs.** Requires `unzip` and `xmllint` on `PATH`, plus network access to GitHub: `make documentation` (this runs `update-docs` — download + extract the release JAR — followed by `build-docs` and the `DocsGenerator` run). Review the diff in `docs/patterns.json` and `docs/description/*` for new/removed/renamed rules and stale fixture references.
3. **Restore and build:** `make configure && make build-all` (or just `make build` for the Analyzer only).
4. **Build the Docker image**: `docker build -t codacy-sonar-csharp:local .`
5. **Run `codacy-plugins-test` locally** before pushing — clone https://github.com/codacy/codacy-plugins-test and run the relevant DockerTest commands (this repo's CI passes `run_multiple_tests: true`, i.e. it exercises the `multiple` tests in `docs/multiple-tests/` in addition to the standard pattern/json tests) against your local image tag.

@codacy-production codacy-production Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM RISK

Suggestion: Clarify how to target the local image and mention the required toolchain. Since this repo is 'no-Scala', agents might prefer the Dockerized runner for codacy-plugins-test to avoid installing sbt. It is recommended to specify that the DOCKER_IMAGE environment variable should be set to the local tag (e.g., DOCKER_IMAGE=codacy-sonar-csharp:local) and mention that codacy-plugins-test can be run via its own Docker image.

6. **Iterate on failures**, re-running only the relevant DockerTest command after each fix.
7. **Commit** the version bump(s) together with the regenerated `docs/` files in one change.
8. **Push and open a PR.** CI runs `checkout_and_version` -> `publish_local` (docker build) -> `plugins_test` -> `publish_docker` (master only) -> `tag_version`.
9. **Poll the PR's real CI checks until they all pass — local validation is NOT the finish line.** After every push, run `gh pr checks <pr-url>` and keep re-polling (short sleep while any check is `pending`) until all checks finish. If a check fails, fetch its actual log (CircleCI API/UI for the failing job — don't guess), find the true root cause, fix it, push again (never `--no-verify`, never force-push), and re-poll. Repeat until every check is green. **The CI environment's toolchain can differ from your local one**, so a clean local run does not guarantee CI passes. Only stop iterating when every check passes, or you hit a genuine product/infra decision that needs a human — in which case explain it in the PR rather than guessing.

### 4. Common failure modes and fixes

| Symptom | Likely cause | Fix |
|---|---|---|
| `make update-docs` fails to download the JAR | Wrong/nonexistent version string, or the release asset naming changed upstream | Verify the exact tag and asset filename on the [sonar-dotnet releases page](https://github.com/SonarSource/sonar-dotnet/releases) |
| `pattern`/`json` DockerTest fails | Rule renamed/removed/added upstream between versions | Re-run `make documentation`; confirm the change matches the upstream changelog for that SonarC# release |
| `multiple` DockerTest fails on a specific fixture folder | Expectations in `docs/multiple-tests/` stale for new analyzer behavior | Regenerate/update the expected results to match the new (verified correct) output |
| Docker build fails after a .NET bump | `TargetFrameworks` in `.csproj` files not updated to match the new `DOTNET_VERSION` base image | Keep `Analyzer.csproj`/`DocsGenerator.csproj` `TargetFrameworks` and the Dockerfile's `DOTNET_VERSION` in sync |
| CI `publish_docker`/`tag_version` don't run on your branch | Expected — gated to the default branch (`master`) only | Nothing to fix |

### 5. Definition of done

- Version bump(s) reflected in `src/Analyzer/Analyzer.csproj` (and `.circleci/config.yml` orbs, if in scope).
- Generated docs (`docs/patterns.json`, `docs/description/*`) regenerated via `make documentation` and committed, with fixture inconsistencies resolved.
- Local `make configure && make build-all` succeeds.
- Docker image builds successfully (`docker build -t codacy-sonar-csharp:local .`).
- `codacy-plugins-test` DockerTest commands all pass locally against the freshly built image.
- **After pushing and opening/updating the PR, every CI check on it is green.** Poll `gh pr checks <pr-url>` and iterate on any failure (fetch the real CI log, fix, push, re-poll) until all pass — a passing local build is not sufficient, because the CI toolchain can differ from your local one (see step 9).

## Docs

[Tool Developer Guide](https://support.codacy.com/hc/en-us/articles/207994725-Tool-Developer-Guide)
Expand Down