Proposed approach for build steps in deps which are not in make node

[mhdawson]

Basic principles

1. Pulling specific versions of tools to to build dependencies from npm is ok
   npm guarantees immutability, and guarantees to give you the same thing for a given version

2. Code that gets pulled into Node.js deps

• Ideally source code is built/transformed in Node.js build (make node)

• Should be able to build from canonical source

  • Project should have copy of canonical source, GitHub is not immutable, and no guarantee that is is available later
  • Project should be able to float patch on dep if needed
    • Should be able to reproduce same result
      • Need to know and have access to specific versions of tools
    • No dynamic downloads

• Couple of categories that are currently worrying in terms of these principles

  • WASM blobs
  • Minified JavaScript
  • TypeScript

Proposed approach for build steps/transforms that cannot done in make node

• Source for transform/build steps outside of make node is is location/repo under the control of the Node.js project
  • Could be:
    • deps subdirectory
    • fork of repo
    • copy of tarball
    • etc.
  • updater will store copy of canonical source in one of the above, build will be done from versioned copies under control of project.
• All build/transform steps run in Docker container
• Docker container contains all tools, etc., needed other than npm dependencies
  • We store copy of container in location under control of project
  • We store copy of dockerfile used to build container along with container
• Build step run by updater uses known container, known canonical source

Next Steps:

• Figure out how we would store containers
• Figure out how we would store tarballs
• Look at undici and outline how we would implement proposed approach above

[mhdawson]

From discussion in last session related to auditing build deps with @marco-ippolito, and earlier discussion with @richardlau as well.

[mhdawson]

@nodejs/security-wg, this has been on the agenda a few times, we assume there are no concerns/objections if we don't hear by next week and move to the next step of PR's the the basic principles into Node.js doc to get broader feedback.

[mhdawson]

Looks like we should be able to use ghcr.io to store container images. https://github.com/features/packages, https://docs.github.com/en/packages/learn-github-packages/introduction-to-github-packages

[mhdawson]

This repo shows some experimentation on createing a repo to generate wasm builder images and storing them in ghcr.io

https://github.com/mhdawson/node-wasm-build/

There is a GitHub action to build and publish. Together with the Dockerfile it tries to capture/keep all key data about what went into the build.

[mhdawson]

Next step will be to look at using the image created to build undici, llhttp instead of building the docker container on the fly as part of the builds for undici, lltttp

[marco-ippolito]

I agree with the approach, we would need to add to the source code on node a folder with the dockerfile (I assume its going to be one that covers all the dependencies but there could be more) and use it in the github action where we do the dependency update