perf(nodejs): reduce Docker image file count with nft static analysis#7079
Open
rochdev wants to merge 17 commits into
Open
perf(nodejs): reduce Docker image file count with nft static analysis#7079rochdev wants to merge 17 commits into
rochdev wants to merge 17 commits into
Conversation
Build the Next.js app during base image creation rather than at weblog build time, since dd-trace is loaded via NODE_OPTIONS at runtime and has no compile-time coupling to the app. This avoids rebuilding on every weblog image update. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the monolithic aws-sdk v2 (~80k files) with three targeted v3 packages (@aws-sdk/client-kinesis, client-sns, client-sqs). The smaller dependency tree reduces Docker image extraction time significantly since extraction cost scales with file count rather than compressed size. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
|
|
This comment has been minimized.
This comment has been minimized.
Add cleanup-node-modules.sh, called at the end of every base image bun install, which removes: - /root/.bun install cache (doubles file count for zero runtime benefit) - .map, .ts, .md, .eslint* files from all packages - test/, examples/, docs/, benchmark/ directories - devDependencies and their orphaned transitives (eslint family, ES polyfill shims, rambda) - es-abstract year directories 2015–2022 (only 2023 is needed by the one remaining consumer, es-aggregate-error via tedious) - @types packages (JavaScript images only) Result: 38,652 → 11,312 total files in express4 base image (−71%). The script includes a maintenance comment explaining how to audit the hardcoded removal list when dependencies change. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bun add in install_ddtrace.sh recreates /root/.bun, adding ~17k files to every weblog image layer. Remove it immediately after the install. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rochdev
changed the title
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…mage `next experimental-compile` compiles lazily at request time and requires source files at runtime. `next build` precompiles all routes, making the src/ directory safe to remove from the final image. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevents Next.js from statically caching route handlers, ensuring dd-trace instruments every request as intended by these tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… prerender next build statically prerenders routes that don't use request data. The healthcheck route requires dd-trace at runtime, so it must run per-request, not be cached from a build-time execution without dd-trace. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
dd-trace instruments at the HTTP transport level so it captures every request regardless of static vs dynamic rendering. force-dynamic on the layout forces React server-component streaming on every request, which breaks older dd-trace versions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
skipLibCheck avoids re-type-checking .d.ts files in node_modules, which is the bulk of tsc's work given the large number of declaration files kept for compilation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Switches all five Node.js weblog base images from the manual cleanup-node-modules.sh pruning script to @vercel/nft, which uses static analysis from the app entry point to keep only files reachable at runtime. This is more principled and more aggressive: ~4,500 files retained vs ~8,700 with the cleanup script (~48% fewer). Key changes: - Add nft-prune.mjs: calls nodeFileTrace() then deletes unreachable files, including itself (not imported by the app) - --keep-types flag preserves node_modules/@types for the TypeScript weblog, which needs declarations for tsc at final-image build time - App source is now COPY'd before bun install so nft has an entry point; bun install and nft-prune run in the same RUN layer so deleted files never appear in the image - Delete cleanup-node-modules.sh, which is no longer used Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rochdev
changed the title
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…s route Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 851b7f1010
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
nft traces binary targets but not the .bin symlinks that point to them. Commands like `next start` use ./node_modules/.bin/next, which would be deleted without this. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Only remove actual caches (/root/.bun, .next/cache). Lockfiles, source files, and config files are harmless and could be useful in the weblog. Empty directory cleanup after pruning is also unnecessary. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…uild-only dirs nft cannot trace dynamic requires in next/dist/compiled (e.g. webpack-lib loaded by config-utils.js at startup). Keep the entire compiled/ directory then remove known build-only entries (experimental React variants, babel, terser) to recover ~19 MB / 473 files. Add --keep-dir flag to nft-prune.mjs for preserving directories regardless of static reachability. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
*babel* glob also matched @babel/runtime which config-utils.js requires at startup. Only delete the build-only babel and babel-packages entries. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reduces the file count in Node.js base images using @vercel/nft static analysis. nft traces the import graph from the app entry point and deletes everything in
node_modulesthat is statically unreachable, giving ~4,500 files per image vs ~21,600 from a rawbun install(−79%).nft works at two levels simultaneously: it eliminates entire unreachable packages (eslint and its transitive deps,
@typesin JS images, unused AWS SDK v3 packages, etc.) and within kept packages it retains only the specific files that are imported, not the full package tree.How it works
A single
nft-prune.mjsscript callsnodeFileTrace()from@vercel/nft, then deletes everynode_modulesfile not in the result.@vercel/nftis added as a devDependency so it installs withbun installand is then pruned away by its own trace (the app never imports it). App source isCOPY'd into the base image beforebun installso nft has an entry point to trace from; both steps run in the sameRUNlayer so deleted files never appear in the image.The script accepts a
--keep-typesflag used byexpress4-typescript, which preservesnode_modules/@types— needed becausetscscans for type declarations dynamically rather than importing them statically.Other changes
NODE_OPTIONSat runtime.Test plan
build-nodejs-base-imageslabel--keep-typeskeeps@typesandtypescript)🤖 Generated with Claude Code