Performance Working Group

Charter

Mission Statement

The Express Performance Working Group establishes a framework for evaluating performance changes in Express, providing consistent measurement criteria, tooling, and guidance to ensure performance-related decisions are made with data and context.

Core Philosophy

We focus on evaluation, not prescription. Rather than dictating how to write fast code, we:

• Define how Express evaluates performance work
• Measure whether changes are meaningful
• Ensure performance PRs are evaluated consistently and in context

Responsibilities

Performance Evaluation Framework

• Establish evaluation criteria (latency, throughput, memory usage)
• Define what constitutes "meaningful" performance changes
• Create standards for testing under various conditions (concurrency, payload size, middleware stack)

Tooling & Infrastructure

• Tooling: see Perf CLI Readme.md
• Provide environments and tools to make measurements possible
• Help contributors run performance tests and interpret results
• Support dependency version overrides for testing performance across different Express versions
• Maintain two categories of tests:
  • Focused unit benchmarks (in package repos)
  • E2E load tests of realistic applications (in express repo)

Key Questions We Answer

• Does this proposed change meaningfully improve performance under realistic load?
• By how much? (p95 latency, req/sec, CPU usage)
• Under what conditions?

Ecosystem Collaboration

• Work with Node.js core on unlocking performance improvements
• Maintain relationships with monitoring providers through OSS programs
• Support public benchmark maintenance

Current Initiatives

Initiative	Champion	Status	Links
Setup Charter	@wesleytodd	In progress	#3

Members

The Performance WG is made up of volunteers, you do not need to be a member to participate, but once you participate please open a PR to add youself here.

Two teams exist for mentioning the group and managing access:

• @expressjs/perf-wg: everyone participating in the WG has write access to this repo
• @expressjs/perf-wg-captains: the repository captians as per Express project guidelines

Team Members

• Wes Todd (Captain)
• Chris de Almeida
• Jean Burellier
• Sebastian Beltran
• Ulises Gascón
• Murat Kirazkaya
• Andre Ferreira
• Luke Lucas
• Jefferson Rios
• Rafael Gonzaga

Meetings

The Performance Working Group meets bi-weekly (see meeting issues). The meeting is open to the public. The agenda and meeting notes are published in this repository. You can find the calendar entries in the OpenJS Foundation calendar.

Offline Discussions

The Performance Working Group uses GitHub issues for offline discussion. The discussions are open to the public and anyone may participate. Also, the group uses the channel #express-perf-wg in the OpenJS Foundation Slack for realtime discussions.

How to Run the Load Tests

Requirements

• Node.js v20 or newer
• Docker for containerized runners
• Load cli's on your PATH
  • autocannon for local runner (optional, can be used via PATH="$(pwd)/node_modules/.bin:${PATH}" npm run test:load)
  • wrk2 for local runner (optional)

Quick Start

1. Install dependencies

   node --run setup

2. Run basic test with vanilla Node.js

   node --run test:load

CLI Usage

CLI Options

The load command supports the following options:

Option	Short	Default	Description
--cwd	-	Current directory	Working directory
--runner	-u	@expressjs/perf-runner-local	Runner module to use
--repo	-	https://github.com/expressjs/perf-wg.git	Git repository URL
--repo-ref	-	master	Git branch/tag to use
--test	-t	@expressjs/perf-load-example	Test module to run
--node	-n	lts_latest	Node.js version to use
--duration	-d	60	Test duration in seconds
--overrides	-o	-	JSON string with package overrides
--config	-c	-	JSON string with package overrides
--[no-]write	-	-	Write results file to disk
--output	-	/results/result-<timestamp>.json	Output file path for results (implies write)
--force-rebuild	-	-	For use with docker based runners, forces rebuilding container

Basic Commands

# Run with default local Node.js runner
node --run load -- --test="@expressjs/perf-load-example"

# Run with specific runner
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid"

# Run with overrides
node --run load -- --test="@expressjs/perf-load-example" --overrides='{"express":"5.0.0"}'

# Run with custom duration (default: 60 seconds)
node --run load -- --test="@expressjs/perf-load-example" --duration=30

# Run with specific runner and overrides
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid" --overrides='{"express":"5.0.0"}'

# Run with specific runner, overrides, and custom duration
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid" --overrides='{"express":"5.0.0"}' --duration=120

# Run with specific runner and overrides and pass api key env
NSOLID_SAAS="XXX" node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid" --overrides='{"express":"5.0.0"}'

# Compare results
node --run compare -- result-A.json result-B.json

Runner-Specific Examples

# Vanilla Node.js (baseline performance)
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-vanilla"

# Vanilla Node.js with --force-rebuild
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-vanilla" --force-rebuild

# N|Solid with built-in monitoring
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid"

# Node.js with Datadog APM (when available)
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-datadog"

# Node.js with New Relic APM (when available)
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-newrelic"

Compare latest Express with a PR branch

Run benchmark against latest Express

node --run load -- --test="@expressjs/perf-load-extended-query" --overrides='{"express":"latest"}' --output=latest.json

Run benchmark against a PR branch

PR from format:

<owner>/<repo>#<branch>

Example:

node --run load -- --test="@expressjs/perf-load-extended-query" --overrides='{"express":"GroophyLifefor/express#no-store-no-etag"}' --output=pr.json

Compare results

node --run compare -- latest.json pr.json

Example output

Comparison: B Wins
==================
{
  diff: { rps: '-6.84%', throughput: '-6.83%', latency: '9.11%' },
  a: {
    avgRPS: 17562.6,
    avgThroughput: 5918720,
    avgLatency: 5.03,
    status: { '200': { count: 1053729 } }
  },
  b: {
    avgRPS: 18851.67,
    avgThroughput: 6352725.34,
    avgLatency: 4.61,
    status: { '200': { count: 1131043 } }
  }
}

GitHub Actions

• The PR workflow compares the pull request's base and head commits on Node.js 18 and 26. Both comparison reports appear in the Summary job.
• When calling the reusable Load Test workflow, gitRef, node, and test are optional and default to main, 24, and @expressjs/perf-load-example, respectively.
• The reusable Compare Load Tests workflow defaults gitRef to main. It requires path, baselineId, testId, and node, and returns the comparison report as Markdown through its summary output.

Passing dependency overrides

The Load Test workflow accepts an optional overrides input containing a JSON object of package names and version specifiers. Leave it empty or use {} to keep the test's declared dependencies. To test an Express commit, use expressjs/express#<commit-sha> and replace <commit-sha> with the full commit SHA.

gitRef selects the revision of perf-wg containing the benchmark code. Use overrides to select the Express version or commit being measured.

From GitHub CLI (gh)

With GitHub CLI installed and authenticated to an account with write access to the repository, run:

# Test a published Express version
gh workflow run load.yml --repo expressjs/perf-wg --ref main \
  -f gitRef=main \
  -f node=24 \
  -f test=@expressjs/perf-load-example \
  -f 'overrides={"express":"5.1.0"}'

# Test an Express commit (replace <commit-sha> before running)
gh workflow run load.yml --repo expressjs/perf-wg --ref main \
  -f gitRef=main \
  -f node=24 \
  -f test=@expressjs/perf-load-example \
  -f 'overrides={"express":"expressjs/express#<commit-sha>"}'

In Bash, keep the single quotes around the overrides=... argument so that the JSON double quotes reach GitHub intact. --ref selects the branch or tag containing the workflow definition; gitRef selects the benchmark revision checked out by that workflow. Change --repo when running in a fork.

From the GitHub Actions website

1. Open the repository's Actions tab and select Load Test.

2. Click Run workflow and select the branch containing the workflow, normally main.

3. Set gitRef to main, node to 24, and test to @expressjs/perf-load-example, or choose another benchmark configuration.

4. In overrides, paste a JSON object such as one of these:

   Selection	Value for the overrides field
   Published version	{"express":"5.1.0"}
   Commit	{"express":"expressjs/express#<commit-sha>"}
   Multiple packages	{"express":"5.1.0","qs":"6.13.0"}

   Keep the JSON double quotes, but omit the surrounding shell single quotes and the overrides= prefix used in the CLI examples. Replace <commit-sha> with an actual full SHA when using the commit example.

5. Click Run workflow to start the load test.

Manual runs require write access, and the workflow must exist on the default branch. See GitHub's manual run guide.

Results appear in the run's summary, with JSON files under Artifacts. Each run tests one dependency selection. To compare versions or commits, run once for each selection with the same node and test, then pass their run IDs to Compare Load Tests.

Programmatic Usage

Basic Import and Usage

// Import specific runner
import vanillaRunner from '@expressjs/perf-runner-vanilla';
import nsolidRunner from '@expressjs/perf-runner-nsolid';

// Run single test
const results = await vanillaRunner({
  test: '@expressjs/perf-load-example',
  overrides: { express: '5.0.0' }
});

console.log(`Requests/sec: ${results.clientResults.requests.mean}`);

Performance Comparison Script

import vanillaRunner from '@expressjs/perf-runner-vanilla';
import nsolidRunner from '@expressjs/perf-runner-nsolid';

async function compareRuntimes() {
  const testConfig = {
    test: '@expressjs/perf-load-example',
    overrides: { express: 'latest' }
  };

  console.log('Running performance comparison...');

  // Run tests in parallel
  const [vanillaResults, nsolidResults] = await Promise.all([
    vanillaRunner(testConfig),
    nsolidRunner(testConfig)
  ]);

  // Compare results
  const vanillaRPS = vanillaResults.clientResults.requests.mean;
  const nsolidRPS = nsolidResults.clientResults.requests.mean;
  const overhead = ((vanillaRPS - nsolidRPS) / vanillaRPS * 100).toFixed(2);

  console.log('Performance Comparison:');
  console.log(`Vanilla Node.js: ${vanillaRPS.toFixed(0)} req/sec`);
  console.log(`N|Solid:        ${nsolidRPS.toFixed(0)} req/sec`);
  console.log(`Overhead:       ${overhead}%`);
}

compareRuntimes().catch(console.error);

Advanced Configuration

import { createRunner } from '@expressjs/perf-runner-shared';

// Create custom runner configuration
const customRunner = createRunner({
  type: 'custom',
  runtime: 'node.js',
  apm: 'custom-monitoring',
  capabilities: ['profiling', 'tracing'],
  env: {
    RUNTIME_TYPE: 'custom',
    CUSTOM_CONFIG: 'enabled'
  }
});

// Use with abort controller for cancellation
const controller = new AbortController();
const results = await customRunner({
  test: '@expressjs/perf-load-example',
  signal: controller.signal
});

Available Runners

Runner	Description	Use Case	Status
@expressjs/perf-runner-local	Local Node.js	Default	Available
@expressjs/perf-runner-vanilla	Standard Node.js runtime	Baseline performance measurements	Available
@expressjs/perf-runner-nsolid	N	Solid runtime with built-in monitoring	Enterprise monitoring capabilities
@expressjs/perf-runner-datadog	Node.js + Datadog APM agent	Datadog APM overhead analysis	Help Wanted
@expressjs/perf-runner-newrelic	Node.js + New Relic APM agent	New Relic APM overhead analysis	Help Wanted

Performance Analysis Examples

CLI-based Comparison

# Step 1: Run baseline test
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-vanilla"
# Output: results/vanilla-result-123456789.json

# Step 2: Run N|Solid test  
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid"
# Output: results/nsolid-result-123456790.json

# Step 3: Compare results
node --run compare -- results/vanilla-result-123456789.json results/nsolid-result-123456790.json

Programmatic Analysis

import fs from 'node:fs/promises';

async function runPerformanceAnalysis() {
  // Test configuration
  const tests = [
    { name: 'Express Hello World', test: '@expressjs/perf-load-example' },
    { name: 'Express with Query', test: '@expressjs/perf-load-extended-query' }
  ];

  const runners = [
    { name: 'Vanilla', runner: vanillaRunner },
    { name: 'N|Solid', runner: nsolidRunner }
  ];

  const results = {};

  // Run all combinations
  for (const test of tests) {
    results[test.name] = {};
    
    for (const { name, runner } of runners) {
      console.log(`Running ${test.name} with ${name}...`);
      
      const result = await runner({ test: test.test });
      results[test.name][name] = {
        requestsPerSec: result.clientResults.requests.mean,
        latencyP95: result.clientResults.latency.p95,
        memoryUsage: result.serverMetadata.memory
      };
    }
  }

  // Save comprehensive report
  await fs.writeFile('performance-report.json', JSON.stringify(results, null, 2));
  console.log('Performance analysis complete. Results saved to performance-report.json');
}

Runner Development

To create a new runner, extend the shared runner infrastructure:

// packages/runner-custom/index.mjs
import { createRunner } from '@expressjs/perf-runner-shared';

export default createRunner({
  type: 'custom',
  runtime: 'node.js',
  apm: 'custom-apm',
  capabilities: ['custom-profiling'],
  env: {
    RUNTIME_TYPE: 'custom',
    CUSTOM_SETTING: 'enabled'
  }
});

Code of Conduct

The Express Project's CoC applies to this repo.