Solution for running/packaging native modules for Electron

[jondo2010]

While attempting to integrate my Electron app using rules_node, I've run into a roadblock with dependent modules that have a native build portion, in particular grpc.

At first I thought I could patch the call to yarn with some flag telling it to "build/install for Electron", which would result in a yarn_modules repository specifically built for Electron, but it seems this is not possible.

Do you have any ideas about how this could be implemented?

[pcj]

Hi @jondo2010 not sure I'm understanding the problem yet. You're not able to use grpc as a yarn dependency? If you can put together a minimal example repo with grpc/electron to iterate on that would be helpful. I'd like to get this working.

[pcj]

Maybe grpc/grpc#8887 is related.

[jondo2010]

Hi @pcj, awesome that you're keen to support! I put together a minimal example here

With the sample output below, you can see that there is a mismatch with the gRPC binary module portion. It needs to be compiled against the correct version of Electron, and this is different than what yarn installs.

$ bazel run '//:test_bin'
INFO: Analysed target //:test_bin (2 packages loaded).
INFO: Found 1 target...
Target //:test_bin up-to-date:
  bazel-bin/test_bin
INFO: Elapsed time: 73.412s, Critical Path: 4.63s
INFO: Build completed successfully, 5922 total actions

INFO: Running command line: bazel-bin/test_bin
stderr: App threw an error during load

stderr: Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v1.8-linux-x64-glibc
Found: [node-v59-linux-x64-glibc]
This problem can often be fixed by running "npm rebuild" on the current system
Original error: Cannot find module '/home/john/.cache/bazel/_bazel_john/8afb8bb0ba5daba4f60180e50fbdc72a/execroot/rules_node_grpc_electron/bazel-out/k8-fastbuild/bin/test_bin_files/node_modules/grpc/src/node/extension_binary/electron-v1.8-linux-x64-glibc/grpc_node.node'
    at Object.<anonymous> (/home/john/.cache/bazel/_bazel_john/8afb8bb0ba5daba4f60180e50fbdc72a/execroot/rules_node_grpc_electron/bazel-out/k8-fastbuild/bin/test_bin_files/node_modules/grpc/src/grpc_extension.js:44:17)
    at Object.<anonymous> (/home/john/.cache/bazel/_bazel_john/8afb8bb0ba5daba4f60180e50fbdc72a/execroot/rules_node_grpc_electron/bazel-out/k8-fastbuild/bin/test_bin_files/node_modules/grpc/src/grpc_extension.js:54:3)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/john/.cache/bazel/_bazel_john/8afb8bb0ba5daba4f60180e50fbdc72a/execroot/rules_node_grpc_electron/bazel-out/k8-fastbuild/bin/test_bin_files/node_modules/grpc/src/client.js:38:12)
A JavaScript error occurred in the main process
Uncaught Exception:
Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v1.8-linux-x64-glibc
Found: [node-v59-linux-x64-glibc]
This problem can often be fixed by running "npm rebuild" on the current system
Original error: Cannot find module '/home/john/.cache/bazel/_bazel_john/8afb8bb0ba5daba4f60180e50fbdc72a/execroot/rules_node_grpc_electron/bazel-out/k8-fastbuild/bin/test_bin_files/node_modules/grpc/src/node/extension_binary/electron-v1.8-linux-x64-glibc/grpc_node.node'
    at Object.<anonymous> (/home/john/.cache/bazel/_bazel_john/8afb8bb0ba5daba4f60180e50fbdc72a/execroot/rules_node_grpc_electron/bazel-out/k8-fastbuild/bin/test_bin_files/node_modules/grpc/src/grpc_extension.js:44:17)
    at Object.<anonymous> (/home/john/.cache/bazel/_bazel_john/8afb8bb0ba5daba4f60180e50fbdc72a/execroot/rules_node_grpc_electron/bazel-out/k8-fastbuild/bin/test_bin_files/node_modules/grpc/src/grpc_extension.js:54:3)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/john/.cache/bazel/_bazel_john/8afb8bb0ba5daba4f60180e50fbdc72a/execroot/rules_node_grpc_electron/bazel-out/k8-fastbuild/bin/test_bin_files/node_modules/grpc/src/client.js:38:12)

[pcj]

OK seeing same error. Electron alone seems to work. Uncertain if node version matters.

Have you seen this working outside of bazel? (probably using electron-rebuild)

[pcj]

OK I think I have this working. Here's what the yarn_modules rule looks like now:

yarn_modules(
    name = "yarn_modules",
    deps = {
        "electron": "1.8.2-beta.4",
        "electron-rebuild": "1.7.3",
        "grpc": "^1.8.4"
    },
    post_install = ["{node}", "node_modules/electron-rebuild/lib/src/cli.js"],
)

1. Need to require electron-rebuild
2. Need to invoke a post-install command to rebuild native libs. The {node} pseudo-syntax will replace the actual path of the node exe bazel is using.

Also, electron-rebuild pulls in rxjs, to which @gregmagolan added a BUILD.bazel file fairly recently (first time I've seen a bazel build file in an npm module!). This causes issues with package boundaries, so I added in a built-in post-install step to remove those from the installed node_modules tree.

I'll push up a PR, let me know what you think.

[alexeagle]

If you just reference the rxjs directory as a workspace, it may resolve the issue since Bazel will no longer descend into the directory. But we'll also maybe run into incompatibility between the node just rules...

[pcj]

Thanks @alexeagle that is a good point.

[jondo2010]

Awesome, this totally works! Thanks for the quick turnaround. I've discovered a few other small gotchas with node_binary that I'll post a PR up for as well.