Skip to content

vfs: provider-based architecture - #2

Merged
mcollina merged 6 commits into
vfsfrom
vfs-provider
Jan 28, 2026
Merged

vfs: provider-based architecture#2
mcollina merged 6 commits into
vfsfrom
vfs-provider

Conversation

@mcollina

@mcollina mcollina commented Jan 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds provider-based architecture with VirtualProvider base class
  • Implements MemoryProvider for in-memory file systems
  • Implements SEAProvider for Single Executable Application assets
  • Exposes node:vfs module for creating virtual file systems
  • Removes backward compatibility methods (addFile, addDirectory, etc.) in favor of standard fs API (writeFileSync, mkdirSync, etc.)
  • Adds setReadOnly() to make MemoryProvider immutable after setup
  • Fixes router.js to handle root mount point (/) correctly

Test plan

  • All VFS tests pass (tools/test.py "test/parallel/test-vfs-*")

cc @Qard

@mcollina
vfs: add provider-based architecture and node:vfs module
956295e 
Redesign VFS with a Provider-based architecture:

- Add VirtualProvider base class defining the provider interface
- Add MemoryProvider for in-memory file storage with full read/write
- Add SEAProvider for read-only access to SEA assets
- Add VirtualFileSystem class as thin wrapper delegating to providers
- Add VirtualFileHandle and MemoryFileHandle for file operations
- Register node:vfs as a builtin module (scheme-only access)
- Add comprehensive API documentation in doc/api/vfs.md

The provider interface supports:
- Essential primitives: open, stat, readdir, mkdir, rmdir, unlink, rename
- Derived methods: readFile, writeFile, exists, copyFile, etc.
- Capability flags: readonly, supportsSymlinks
- Dynamic content providers (functions called on each read)
- Lazy directory population
@mcollina
vfs: remove backward compat methods, use standard fs API
d207441 
Remove backward compatibility methods (addFile, addDirectory,
addSymlink, has, remove) from VirtualFileSystem. Users should now use
the standard fs-like API (writeFileSync, mkdirSync, symlinkSync,
existsSync, unlinkSync).

For dynamic content and lazy directories, use the provider methods:
- provider.setContentProvider(path, fn) for dynamic file content
- provider.setPopulateCallback(path, fn) for lazy directory population

Also adds:
- MemoryProvider.setReadOnly() to make provider immutable after setup
- Fix router.js to handle root mount point (/) correctly
@mcollina
vfs: address review comments
cef55ed 
- Remove setContentProvider() and setPopulateCallback() methods
- Remove getContent() backward compat wrapper from MemoryProvider
- Remove section separator comments from memory.js and sea.js
- Remove related tests for dynamic content and lazy directories
- Update documentation to remove the removed methods
Qard
Qard approved these changes Jan 28, 2026
View reviewed changes

@Qard Qard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool! The mounting into the fs module thing is neat. I avoided that in my own branch where I was experimenting with this though as I was a bit uncertain about the possible security implications. My thinking was that code that wanted to accept provider-based file system access configurability could just use dependency-injection to accept either a vfs instance or just do thing(require('fs')) to pass the raw fs API and it wouldn't need to know the difference--that's why I was going for API equivalency.

Admittedly though, being able to overlay a mount point over the true filesystem is pretty cool. I wonder though if it might also be a bit confusing that it's not actually a real OS-level mount. 🤔

Comment thread doc/api/vfs.md
@mcollina
vfs: fix lint errors
38f5110 
- Fix template string in documentation examples
- Remove unused imports (isVirtualFd, VFS_FD_BASE, kEntries, isWrite)
- Add createENOTDIR to top-level imports, remove duplicate require
- Use DateNow from primordials instead of Date.now()
- Add MathMin, Boolean to primordials at top of sea.js
- Add parentheses to arrow function parameter
@mcollina
vfs: fix lint errors
37d40a2 
- Use internal error codes instead of new Error()
- Add null-prototype to async return objects
- Fix JSDoc for abstract methods (use @throws instead of @returns)
- Add createENOTEMPTY error creator
- Fix markdown lint (Node.js's -> the Node.js)
@mcollina
vfs: remove public createSEA()
2364402 
SEA assets are automatically mounted at /sea when running as a
Single Executable Application. No user action is required.
@mcollina
mcollina merged commit 2364402 into vfs Jan 28, 2026
14 of 23 checks passed
@jsumners

jsumners commented Jan 28, 2026

Copy link
Copy Markdown

I avoided that in my own branch where I was experimenting with this though as I was a bit uncertain about the possible security implications.

Check out https://pkg.go.dev/os@go1.25.6#Root (https://medium.com/@ajitem/secure-filesystem-access-in-go-1-24-introducing-os-root-dcb031732516).

This is a nice feature. Could be really helpful for testing. The equivalent in Go really helps there.

mcollina pushed a commit that referenced this pull request Jun 7, 2026
@ndossche
src: fix edge case when deflateInit2() fails with Z_VERSION_ERROR
458c37b 
This function call can fail with `Z_VERSION_ERROR` if the compiled
library vs loaded library mismatched in version number or in
stream structure size.
In those cases, zlib doesn't initialize the `strm_.msg` field to
null. Therefore, when a `CompressionError` object is created via
`ErrorForMessage()`, it can read a stale or uninitialized `strm_.msg`
pointer that will cause a crash.

Example ASAN report:
```
AddressSanitizer: SEGV on unknown address
    #0 __strlen_avx2
        string/../sysdeps/x86_64/multiarch/strlen-avx2.S:76
    #1 strlen (/work/node/out/Debug/node+0x1a42ab7)
    #2 v8::(anonymous namespace)::StringLength(char const*)
        /work/node/out/../deps/v8/src/api/api.cc:7581:16
    #3 v8::(anonymous namespace)::StringLength(unsigned char const*)
        /work/node/out/../deps/v8/src/api/api.cc:7587:10
    #4 v8::String::NewFromOneByte(v8::Isolate*,
        unsigned char const*, v8::NewStringType, int)
        /work/node/out/../deps/v8/src/api/api.cc:7677:3
    #5 node::OneByteString(v8::Isolate*,
        char const*, int, v8::NewStringType)
        /work/node/out/../src/util-inl.h:166:10
    #6 node::(anonymous namespace)::CompressionStream<
        node::(anonymous namespace)::ZlibContext>
        ::EmitError(node::(anonymous namespace)
        ::CompressionError const&)
        /work/node/out/../src/node_zlib.cc:565:7
    #7 node::(anonymous namespace)::CompressionStream<
        node::(anonymous namespace)::ZlibContext>
        ::CheckError()
        /work/node/out/../src/node_zlib.cc:519:5
    #8 node::(anonymous namespace)::CompressionStream<
        node::(anonymous namespace)::ZlibContext>
        ::AfterThreadPoolWork(int)
        /work/node/out/../src/node_zlib.cc:543:10
    #9 node::ThreadPoolWork::ScheduleWork()
        ::'lambda'(uv_work_s*, int)
        ::operator()(uv_work_s*, int) const
        /work/node/out/../src/threadpoolwork-inl.h:57:15
    nodejs#10 node::ThreadPoolWork::ScheduleWork()
        ::'lambda'(uv_work_s*, int)
        ::__invoke(uv_work_s*, int)
        /work/node/out/../src/threadpoolwork-inl.h:48:7
    nodejs#11 uv__work_done /work/libuv-1.51.0/src/threadpool.c:330:5
    nodejs#12 uv__async_io.part.0
        /work/libuv-1.51.0/src/unix/async.c:208:5
```

Signed-off-by: ndossche <nora.dossche@ugent.be>
PR-URL: nodejs#63476
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@Qard Qard Qard approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mcollina @jsumners @Qard