Repository navigation
shared memory for multiple processes on same machine. #8
Description
is it possible to share the libpostal memory between two different nodejs processes calling require('node-postal') on the same machine?
calling require('node-postal') uses a fair chunk of RAM so having two different services on the same machine using node-postal consumes more RAM than maybe it needs to?
I'm assuming that the memory is read-only so would it be possible to share that memory between processes?
Activity
albarrentine commentedon Nov 16, 2016
Not without re-architecting a few parts of libpostal.
require('node-postal')reads the model files from disk and creates the necessary in-memory data structures for libpostal usingmalloc. To share memory between processes in UNIX you're almost always talking aboutmmap. Parts of the said model files like the arrays/matrices should be mmap-able, but because of the way we store numeric types on disk (endian-agnostic, floats/doubles stored as their uint32/uint64 representations and converted on read), and the fact that mmap would only provide access to the raw bytes, it would require quite a bit of refactoring apart from just the I/O functions.In any case, reducing memory should be relatively easy to accomplish at the application level by designating one process per machine to load libpostal and communicate with other processes through a named pipe, a TCP socket, something like ZeroMQ, Redis, etc.