PicklingError on multiprocessing.pool "Can't pickle <class 'zstd.ZstdError'

[hub-il]

Using the zstandard module to compress and decompress data in a multiprocessing context. If there is a zstd.ZstdError thrown when running in a pool such as:

zstd.ZstdError: decompression error: did not decompress full frame

It results in this error:

multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x74ccabe98bb0>'. Reason: 'PicklingError("Can't pickle <class 'zstd.ZstdError'>: import of module 'zstd' failed")'

Seems that there is something in the zstd.ZstdError class that can't be pickled by the multiprossing.pool.

[Peter-Schay]

This workaround solved it for my app:

import sys, zstandard
sys.modules['zstd'] = sys.modules['zstandard']