ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING error when using ESM script to render

[aKzenT]

When trying to use a ESM script with the library I'm getting a ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING error on the line that dynamically imports the script.

A dynamic import callback was not specified. TypeError [ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING]: A dynamic import callback was not specified. at new NodeError (node:internal/errors:399:5) at importModuleDynamicallyCallback (node:internal/modules/esm/utils:89:9) at Http2ServerRequest. ([eval]:1:3021) at Http2ServerRequest.emit (node:events:512:28) at endReadableNT (node:internal/streams/readable:1359:12) at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
at Jering.Javascript.NodeJS.HttpNodeJSService.TryInvokeAsync[T](InvocationRequest invocationRequest, CancellationToken cancellationToken)
at Jering.Javascript.NodeJS.OutOfProcessNodeJSService.TryInvokeCoreAsync[T](InvocationRequest invocationRequest, CancellationToken cancellationToken)
at Jering.Javascript.NodeJS.OutOfProcessNodeJSService.InvokeFromFileAsync[T](String modulePath, String exportName, Object[] args, CancellationToken cancellationToken)

This is the line from the Javascript.NodeJS library minified script that causes the exception:

      a = e.endsWith(".mjs") ? await import("file:///" + e.replaceAll("\\", "/")) : require(e)

The script with "mjs" extension looks very simple:

export default async(data) => {
  console.log("render called");
  return Promise.resolve("Test");
};

The strange thing is also that initially it does seem to work, so some calls are successful, but then it stops working with that error.

Any ideas?

I'm using node 19.7.0 and 7.0.0-beta3 of this library.

[JeremyTCD]

It's not your logic, I noticed this error in a recent CI build.

This library's logic is doing the bare-minimum, just calling import(<file>). From that and the intermittent nature of the issue, it looks like a Node.js problem.

I've found where the error is thrown in Node.js: https://github.com/nodejs/node/blob/0c460518e8c0728294cfbb5313d347a62856a2c5/lib/internal/modules/esm/utils.js#L89.

When I have some time to investigate I will look into it, but it may be a while (several weeks).

If you have spare bandwidth, it would be helpful if you could try creating a repro of the issue (without this library) and reporting the issue to the Node.js team if you manage to.

In the meantime, .mjs should not be used in production with this library.

[aKzenT]

I did try some simple reproductions by trying to import a mjs module from a script evaluated dynamically from the command line like the library does. However even when calling import multiple times, I did not get that error. So I have no idea what exactly triggers this and it does not seem to be a known problem in NodeJS.

One simple workarround that works for me is just wrapping the MJS file in a CJS file like this:

module.exports = async(data) => {
    var render = (await import('./render.mjs')).default;
    return await render(data);
};

One idea you might want to look into is changing the embedded scripts to be ESM as well and then specify the --input-type on the command line (https://nodejs.org/dist/latest-v19.x/docs/api/cli.html#--input-typetype). My limited understanding is that in general it is easier to import a CJS file into ESM than the other way arround.

[JeremyTCD]

Okay, "script evaluated dynamically" meaning using --eval/-e?

Do you mind cloning this repo, adding a test that reproduces the issue and creating a PR? Would be good to have a test case to debug against.

I'm open to converting the embedded scripts to es modules, but will need some way to verify that doing so fixes the issue.

[aKzenT]

I created a PR that reproduces the issue fairly reliable on my PC. However it seems that this only happens infrequently. So I had to do a lot of retries which makes the test very slow.

One thing I noticed in the test protocol though was that even before the exception occurs there are connection resets and retries with the error stating that the connection was closed by the remote host. That makes me wonder whether maybe the error does occur every time, but the library is not fast enough to read the error before the process crashes. Could that be possible?

[JeremyTCD]

Thanks for the PR.

With regard to the "connection closed" retries, dynamic imports are within a try-catch block. If the dynamic import error gets caught some of the time, I don't see how it could get past the try-catch block on other occasions.

Imo it's likely some other error, but might be related? Either fatal (killing the process immediately) or uncaught error. I've just done some reading up, we should add:

process.on('uncaughtException', ...);
process.on('unhandledRejection', ...);

to rule out uncaught errors. Should also check windows event viewer for any further information on fatal errors.

Another thing we can do would be to simplify the HTTP server to its most basic (for invoking .mjs), see if that resolves the issue and add back stuff till we find the offending logic. For example, right now the server polls the .Net process every second (Shared.ts), stuff like that might be affecting the server.

If none of that resolves the issue, we'll have to look into Node.js, crash dumps, etc. Appreciate if you could start on investigations, will try when I have more time!

[thebuilder]

Was just wondering if this was just me. Also getting the error after switching our Vite output to the modern .mjs.
Great that you are already on top of this. 💪 Vite 5 might drop support for .cjs, so getting it working properly for the future would be nice: vitejs/vite#12466 🙏

A dynamic import callback was not specified.
TypeError [ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING]: A dynamic import callback was not specified.
    at new NodeError (node:internal/errors:371:5)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:39:9)
    at IncomingMessage. ([eval]:1:3020)
    at IncomingMessage.emit (node:events:402:35)
    at endReadableNT (node:internal/streams/readable:1343:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

[thebuilder]

Hey @JeremyTCD - Have you had a chance to dig further into this issue?🙏

One simple workarround that works for me is just wrapping the MJS file in a CJS file like this:

module.exports = async(data) => {
    var render = (await import('./render.mjs')).default;
    return await render(data);
};

Have you been using this workaround in production @aKzenT? Any issues?

[JeremyTCD]

Hello Daniel, I must apologize, as I haven't had the opportunity to work on this library recently due to other commitments. I'm looking to bring in new maintainers to help manage the project. If you, or anyone else, are interested in this role, please don't hesitate to let me know.

…

Message ID: ***@***.*** com>

[aKzenT]

Have you been using this workaround in production @aKzenT? Any issues?

So far I did not have any issues with this approach, but it is not in production yet. That being said, I did encounter the issue even during development when using ESM directly, so I'm feeling confident that the workarround works for now. A native solution would of course be nicer.

[thebuilder]

So far I did not have any issues with this approach, but it is not in production yet. That being said, I did encounter the issue even during development when using ESM directly, so I'm feeling confident that the workarround works for now. A native solution would of course be nicer.

Looked like a good solution, but I did a quick test of this approach, and ran into this error in my project:

import { hyphenateHTMLSync } from "hyphen/da/index.js";
         ^^^^^^^^^^^^^^^^^
SyntaxError: Named export 'hyphenateHTMLSync' not found. The requested module 'hyphen/da/index.js' is a CommonJS module, which may not support all module.exports as named exports.
CommonJS modules can always be imported via the default export, for example using:

The error is triggered by loading .cjs, that loads .mjs that then loads another .cjs from node_modules. I did the test by loading the file directly with Node, so not even caused by this library.

Since Vite 4.4.4, they are logging a warning that .cjs support will be removed in version 5.

UPDATE: Solved the issue by changing the import, so it imports the .js file directly. 👍

[thebuilder]

I'm not that confident i .NET, but trying to get the project to run, and looks like it needs some env variables to run the Webpack build task. Right now it fails because env.mode and env.entry is undefined.

[aKzenT]

Unfortunately I just encountered this issue on production even with the workarround of loading ESM from CJS :-(

So it seems that ESM will not work correctly until the loader script is in ESM as well. @thebuilder did you figure out how to build the project?

[thebuilder]

Unfortunately I just encountered this issue on production even with the workarround of loading ESM from CJS :-(

So it seems that ESM will not work correctly until the loader script is in ESM as well. @thebuilder did you figure out how to build the project?

Which issue? I solved the CommonJS imports issue, by changing the import (to what it suggests in the error mesage).

We are currently looking into running pure Node server alongside the .NET project, and having them communicate over localhost HTTP.

[aKzenT]

I mean I got ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING exceptions even when using a CJS wrapper to import my ESM file.