Skip to content

CLAUDE.md

Latest commit

 

History

History
385 lines (260 loc) · 16.1 KB

CLAUDE.md

File metadata and controls

385 lines (260 loc) · 16.1 KB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

CRITICAL: Git Commit Rules

NEVER EVER use --no-verify flag when committing. This bypasses important checks and is lazy and unprofessional. If pre-commit hooks fail:

  1. FIX the issues that the hooks are reporting
  2. Run the hooks locally to verify they pass
  3. Only commit when ALL checks pass cleanly
  4. NO EXCEPTIONS to this rule

Using --no-verify is a sign of poor craftsmanship and will not be tolerated.

CRITICAL: Self-Hosted CI Configuration

The .github/workflows/ directory is GENERATED by cigen itself. NEVER manually edit files in that directory.

  • CI configuration is defined in .cigen/ directory
  • Workflows are generated by running: cargo run -- --config .cigen generate
  • All changes to CI must be made in .cigen/workflows/ YAML files
  • The generated files contain "DO NOT EDIT" warnings at the top
  • To test CI locally: act -j <job-name> --container-architecture linux/amd64

If you need to modify CI behavior:

  1. Edit files in .cigen/workflows/ci/jobs/
  2. Regenerate with cargo run -- --config .cigen generate
  3. Test with act before committing
  4. Commit both the source .cigen/ files AND generated .github/ files together

Working With act

Fast local feedback depends on using the same runner image and artifact flow as GitHub Actions. Follow this recipe:

  • Build the local runner image (defaults to linux/arm64; override with CI_RUNNER_PLATFORM=linux/amd64 if needed):

    ./scripts/build-ci-runner.sh

  • Run jobs with act, mapping runner architecture and storing artifacts locally so download/upload steps succeed:

    act --artifact-server-path tmp/artifacts \
    --pull=false --action-offline-mode \
    --container-architecture linux/arm64 \
    -j fmt
    • --pull=false and --action-offline-mode force Docker to reuse the locally built image instead of hitting Docker Hub.
    • The artifact server directory must exist; act will persist uploaded artifacts there so downstream jobs can download them.
    • Use the shell timeout 600 -- act ... wrapper (or similar) to enforce the 10-minute cap on local runs.

  • To make these options the default, add them to ~/.actrc:

    --artifact-server-path tmp/artifacts
--pull=false
--action-offline-mode
--container-architecture linux/arm64

  • Cleaning up the artifact directory between runs (rm -r tmp/artifacts) prevents stale caches from masking issues.

Rebuild the runner image whenever dependencies change (e.g., adding packages to docker/ci-runner/Dockerfile), then rerun act to validate the workflows end-to-end.

Required Local Validation Workflow

You must follow this exact sequence whenever you touch CI-related code or generated workflows:

  1. Run the job's core command locally (e.g., cargo fmt -- --check, cargo clippy --all-targets --all-features -- -D warnings, or cargo test --workspace). Pick the job that will exercise your change.
  2. Run task ci and ensure it passes with no warnings.
  3. Re-run that same job via act (timeout 600 act ... -j <job-name>), using the local runner image so we mirror GitHub Actions.
  4. Only after the above succeed may you commit and push to GitHub.

Skipping or reordering these steps is not acceptable—they keep our feedback loop fast and avoid 10+ minute CI retries.

Project Overview

cigen is a Rust CLI tool that generates CI pipeline configurations from templates. It uses a plugin-based architecture that currently supports GitHub Actions and CircleCI, with plans to support additional CI providers like Woodpecker CI.

Shell Script Compatibility

CRITICAL: When writing shell scripts or commands (especially for version detection), ensure full compatibility across all systems:

  1. Use POSIX-compliant features only - No GNU-specific flags or modern bash features

  2. Test compatibility with both BSD (macOS) and GNU (Linux) versions of tools

  3. Avoid these common incompatibilities:

    • grep -P (PCRE) - Not available on BSD grep
    • grep -o with complex patterns - Behavior differs between implementations
    • sed -i without backup extension on macOS
    • Modern bash features like [[ conditions or arrays

  4. Preferred patterns:

    • Use grep -E instead of grep -P for extended regex
    • Use simple grep | grep chains instead of complex single patterns
    • Always provide backup extension for sed -i: sed -i.bak (then delete .bak file)
    • Use /bin/sh compatible syntax, not bash-specific

  5. Example of compatible version extraction:

    # Good - works everywhere
ruby --version | grep -o '[0-9]*\.[0-9]*\.[0-9]*' | head -1

# Bad - GNU grep only
ruby --version | grep -oP '\d+\.\d+\.\d+'

Build Commands

# Build the project
cargo build

# Run tests
cargo test

# Format code
cargo fmt

# Lint code (use the same flags as the git hook)
cargo clippy --all-targets --all-features -- -D warnings

# Test validation on the example
cargo run -- --config integration_tests/circleci_rails/ validate

# Test generation on the example
cargo run -- --config integration_tests/circleci_rails/ generate

Rust Code Style

IMPORTANT: Always use inline format variables in Rust. Clippy enforces this with the uninlined_format_args rule.

Examples:

  • ❌ WRONG: println!("{}", variable)
  • ✅ CORRECT: println!("{variable}")
  • ❌ WRONG: format!("{}/{}", workflow_name, job_name)
  • ✅ CORRECT: format!("{workflow_name}/{job_name}")
  • ❌ WRONG: println!("Value: {}", value)
  • ✅ CORRECT: println!("Value: {value}")

Version Management

CRITICAL: NEVER rely on your own knowledge of package versions, tool versions, or dependency versions. They are ALWAYS out of date. You MUST:

  1. ALWAYS search the web to find the latest stable version of ANY package, tool, or dependency
  2. NEVER assume version numbers from memory
  3. ALWAYS verify current versions before adding them to any configuration file
  4. For download URLs or scripts, ALWAYS fetch the latest version dynamically or provide clear instructions for users to check the latest version

Examples of Version Mistakes Made:

  1. Rust version: Initially used 1.83.0 when 1.88.0 was available
  2. Crate versions: Used tracing-subscriber = "0.3.20" which didn't exist (latest was 0.3.19)
  3. Lefthook setup script: Hardcoded version 1.8.4 in the download URL instead of fetching latest

What YOU (Claude) Must Do:

WRONG APPROACH (what I did):

# I wrote this without checking the actual latest version:
LEFTHOOK_VERSION="1.8.4"  # This was me guessing from my outdated knowledge!

CORRECT APPROACH (what I should have done):

  1. First, search the web: "lefthook latest release github 2025"
  2. Find the actual latest version (e.g., maybe it's 1.10.2)
  3. THEN write the code with the correct version:
LEFTHOOK_VERSION="1.10.2"  # After verifying this is the actual latest version

The rule is: I (Claude) must ALWAYS search for the current version before writing ANY version number in code. The user shouldn't have to correct version numbers - I should get them right the first time by searching.

Cargo.toml Dependency Management

CRITICAL: NEVER manually add dependencies to Cargo.toml with version numbers. ALWAYS use cargo add:

# ❌ WRONG: Manually editing Cargo.toml
petgraph = "0.6"  # This version is likely outdated!

# ✅ CORRECT: Using cargo add
cargo add petgraph  # Automatically fetches and adds the latest version

This ensures we always get the latest compatible version and properly updates Cargo.lock.

Code Organization

CRITICAL: Keep files small and modular. As soon as a file approaches 200-300 lines, break it up into modules and smaller files. DO NOT wait for the user to remind you. Proactively refactor large files into:

Releases (GitHub + Docker)

  1. Update versions:
    • Cargo.toml
    • plugins/provider-circleci/Cargo.toml
    • plugins/provider-github/Cargo.toml
  2. Run cargo update -p cigen.
  3. Commit the version bump and any release notes.
  4. Tag with git tag vX.Y.Z (push tag to trigger workflow) or trigger .github/workflows/release.yml manually.
    • The workflow builds archives for macOS/Linux and publishes them via softprops/action-gh-release.
    • It also builds/pushes docspringcom/cigen:<version> and :latest from the repository Dockerfile.
  5. Update docker/ci-runner/Dockerfile (if it depends on the version) and rebuild/push via scripts/build-ci-runner.sh + docker push docspring/cigen-ci-runner:latest.
  • Separate modules for distinct functionality
  • Helper functions in their own files
  • Traits and implementations in separate files
  • Tests in separate test modules

This keeps the codebase maintainable and easier to understand.

Development Approach

CRITICAL: NEVER implement temporary solutions or workarounds. NEVER say "for now" or "this is not ideal but". Always implement the proper, clean, architectural solution from the start. If something needs to be done right, do it right the first time.

CRITICAL: NEVER say "quickly" or "let me quickly" - nothing should be done quickly. Everything requires careful thought and proper implementation. Speed is not a priority; correctness is.

IMPORTANT: Work on one small piece at a time. Do not attempt to build the entire project at once.

  1. Set up foundation first (Cargo.toml, basic CLI with --help and --version)
  2. Establish testing infrastructure before adding features
  3. For each new feature:
    • Write tests first
    • Implement minimal code to pass tests
    • Run cargo fmt and cargo clippy
    • Verify tests pass
    • Commit to git
    • Only then move to the next feature

Small, verifiable chunks prevent errors and ensure steady progress.

Before Marking Tasks Complete

CRITICAL: Before marking ANY task as complete in your todo list, you MUST follow this workflow:

Step 1: Run All CI Checks

task ci

This runs all linters and tests that run on CI, including:

  • cargo fmt --check - Verify code formatting
  • cargo clippy --all-targets --all-features -- -D warnings - Lint checks
  • cargo test --workspace - All tests in workspace
  • Any other CI checks defined in Taskfile

If task ci fails:

  1. Read the error messages carefully
  2. Fix ALL issues reported
  3. Run task ci again
  4. Repeat until it passes cleanly

Step 3: Mark Task Complete

ONLY after this condition is met:

  • ✅ task ci passes with no errors

Then and only then can you mark the task as complete in your todo list.

Why This Matters

This workflow ensures:

  • Quality: Code meets all project standards
  • Consistency: Same checks run locally and on CI
  • No Technical Debt: Issues are fixed immediately, not deferred

NO EXCEPTIONS: This applies to ALL tasks, no matter how small. Good habits compound.

Error Handling

CRITICAL: NEVER silently fail or provide dummy/placeholder behavior:

  1. NO DEFAULT VALUES: Never return placeholder text like "No command specified" or empty defaults when data is missing
  2. FAIL FAST: Use bail!() or panic!() with descriptive error messages when encountering invalid states
  3. EXPLICIT ERRORS: Always provide clear, actionable error messages that explain what went wrong
  4. VALIDATE EARLY: Check preconditions and validate data as early as possible
  5. PRESERVE UNKNOWN DATA: When encountering unknown configurations or step types, preserve them as-is rather than dropping them or converting to defaults

Bad example:

// NEVER DO THIS
command: "echo 'No command specified'".to_string(),

Good example:

// DO THIS INSTEAD
miette::bail!("Invalid step configuration: missing required 'command' field")

CircleCI Reference Implementation

IMPORTANT: The source of truth for all base features is in circleci_config_reference/src/*. This contains the ERB templates from our production Ruby implementation. All features shown there must be supported.

You can also reference /Users/ndbroadbent/code/docspring/lib/tools/generate_circle_ci_config.rb to understand how the Ruby script worked.

Testing Strategy: Add comprehensive test cases as you implement features, based on what you find in the reference implementation. Every feature from the reference should have corresponding tests.

DocSpring Integration vs Public Examples

CRITICAL SECURITY DISTINCTION:

Public Examples (integration_tests/)

  • PUBLIC MIT LICENSED code visible to all users
  • Serves as demonstration of complex, production-ready CI pipeline patterns
  • Must contain NO internal DocSpring information, secrets, or sensitive details
  • Use generic company names, sanitized configurations, example data only
  • This is what users see to learn how to use cigen

Private DocSpring Work (./docspring/)

  • PRIVATE INTERNAL DocSpring monorepo (symlinked)
  • Contains actual DocSpring production CI configuration
  • Work in ./docspring/.cigen/ directory for real DocSpring conversion
  • Source jobs are in ./docspring/.circleci/src/ci_jobs/ and ./docspring/.circleci/src/deploy_jobs/
  • Convert ERB templates to cigen YAML format in ./docspring/.cigen/workflows/
  • Use DocSpring's actual production requirements

NEVER COPY INTERNAL DOCSPRING DETAILS TO PUBLIC EXAMPLES

Use Our Own Tool

The goal is to eventually become 'self-hosting' for our own CI pipeline on GitHub Actions. We already keep turbo.json and other workspace metadata in the repo alongside .cigen/ so that the CLI can evolve toward Turborepo-aware features.

We will start by hand-writing our own GitHub Actions workflow files, but eventually migrate to using cigen to generate our CI configuration.

DocSpring Configuration Location

CRITICAL: The docspring/ directory in this repository is a SYMLINK to a COMPLETELY SEPARATE repository - the DocSpring monorepo. These are TWO SEPARATE REPOS:

  1. cigen repository (/Users/ndbroadbent/code/cigen/) - The tool itself
  2. DocSpring monorepo (/Users/ndbroadbent/code/docspring/) - A separate private repository

The symlink at /Users/ndbroadbent/code/cigen/docspring → /Users/ndbroadbent/code/docspring exists ONLY for convenience during development.

IMPORTANT:

  • The docspring/ directory is in .gitignore - it is IGNORED by git
  • It is NOT a submodule
  • There is NO git relationship between these repositories
  • Changes to files in docspring/ are tracked by the DocSpring repo, not cigen
  • When working on DocSpring configuration, you're editing files in the DocSpring monorepo

When working on DocSpring configuration conversion:

  • The .cigen/ directory is located in the DocSpring monorepo at /Users/ndbroadbent/code/docspring/.cigen/
  • This is on the nathan/cigen-config branch of the DocSpring monorepo
  • All job definitions, commands, and templates for DocSpring should be created there
  • You CAN and SHOULD commit DocSpring changes: Simply cd docspring and commit normally
  • The whole point is developing cigen AND porting DocSpring CI config in tandem
  • DocSpring changes are tracked on its own cigen branch, completely separate from the main cigen repo

Split Configuration

IMPORTANT: CIGen supports a "split config" style where different top-level keys can be refactored into their own files under .cigen/config/, and it all gets merged together. This prevents .cigen/config.yml from becoming unmaintainable.

We MUST use split config for large configuration sections:

  • Source file groups are already split: docspring/.cigen/config/source_file_groups.yml
  • Any other config section that grows beyond ~20 lines should be split into its own file
  • Files in .cigen/config/ are automatically merged with the main config.yml
  • Use descriptive filenames that match the top-level key (e.g., cache_definitions.yml for cache definitions)

Key Concepts

  • The tool no longer relies on the old workspace-specific metadata that was tied to previous experiments. Future monorepo integrations will be driven by the plugin system (e.g., Turborepo) once the orchestration layer is complete.
  • Templates and configuration live in the .cigen/ directory
  • The tool supports plugin-based cache backends and CI provider emitters