change API to get page-level parallelization everywhere

[bertsky]

In #278 we already had the case were the METS got broken due to ctrl+C in a single process. We discussed about parallelization but agreed to defer that topic. Now this is about providing a locking mechanism that allows parallel processing.

Currently it is impossible to run in parallel over a workspace, simply because it is too dangerous: the result of one actor could be lost due to another without any indication of it. This hampers makefilization and similar efforts.

IIUC, getting FS locking in a portable way is tricky, if you want to get it right.

Does anyone have experience or solutions?

[wrznr]

No experience let alone solutions. Sry.

[kba]

Portability isn't of the essence though because I don't foresee the need to support non-POSIX filesystems.

No experience doing this with Python but the principle is simple and https://github.com/dmfrey/FileLock/blob/master/filelock/filelock.py (the accepted answer you linked to) looks like a reasonable solution.

[bertsky]

I'd like to bring in another idea: Maybe file locking is the wrong way to think of page-level parallelization altogether. We have dependencies between successive processors in a workflow, so it does not make much sense to think of running these in parallel (without going into elaborate synchronization mechanisms like semaphores for all pages), except for a very narrow set of circumstances where workflows contain parallel branches. On the other hand, what really can be run independently (at least for most processors and workflows) are the pages within one processing step. Even now, a processor implementation can do that, e.g. by using a multiprocessing.Pool.map() on the workspace.input_files.

And there's the rub really: In the current design, Processor.process() implementations must iterate over input files themselves. The responsibility for parallelization therefore is in the module provider (instead of the base class in core).

So for 3.0, what we should aim at is the following:

1. deprecate Processor.process()
2. create Processor.process_page() as an abstract method, taking a single OcrdFile as input – and change all implementations in the modules accordingly (which is a lot of work, but trivial).
3. create Processor.process_workspace(), iterating over the input files and delegating to process_page() respectively. Here we have the freedom to use a CPU-parallel implementation under user control. (Of course, as in the workspace-parallel scenario, we then have to make sure GPU-enabled processors do not over-allocate the GPU resources and cause memory races; either using a CPU-fallback or retreating to a safe GPU "bottleneck".)

As a side note: One could of course always argue that workspace-parallel execution is much less complicated and ensures maximum throughput without extra design/implementation effort and synchronization overhead. But this does not cover the use-case of a single (large) workspace, or the use-case of minimum latency.

I guess this proposal should go through the spec first?

[kba]

It's a good proposal, while the Processor.process approach with a thin wrapper around the METS was a neat idea at the time, it has run its course and breaking this up will allow us more flexibility. I also agree that iterating over the files from within the processor is not a scalable approach and also leads to unnecessary and error-prone copy-paste boilerplate in the processors. We should also make sure that the process_* methods are implemented in a way that there are ways to augment them at runtime with hooks or a method convention like "processor implements a method _process_page" while the BaseProcessor offers flexible process_page method wrappers.

I guess this proposal should go through the spec first?

Not really, the workspace-processor approach in core is just one implementation and while it will be some work to refactor the processors, they will still follow the same CLI, just that the code of the processors can get rid of a lot of loops and inconsistent implementations across processors. From what I see this change should not result in any changed behavior for users? I will think about it some more though, quite possible I'm missing something here.

[bertsky]

One thing we have to take into account are processors that run on multiple input file groups. For those, process_page will have to take a tuple of OcrdFiles – for the same pageId. Our base implementation process_workspace will have to provide for that. (And if no full matches can be found for some pages, then either skip with an error or provide the processor with a None input.) This is what ocrd-cor-asv-ann-evaluate or ocrd-cis-align do on their own now. But there might be additional concerns in the processor, like MIME type filtering (if there is more than 1 file per page in some fileGrp).

There might even be processors which read multi-page files (like COCO segmentation, TIFF or PDF). For example, https://github.com/OCR-D/ocrd_segment/blob/master/ocrd_segment/import_coco_segmentation.py. See OCR-D/spec#142

[jbarth-ubhd]

My toughts about parallelization:

• would it be possible to parallelize single processing steps automatically within given bounds (CPU, RAM)
• this would make it easier to start multiple workflows at a time
• there could be negative effects: CPU+Disk Cache trashing, RAM bandwidth, slowed down i/o due to disk latency ...
• e. g. Imagemagick-Q16 needs about 2Copies * Pixels * 2BytePerChannel * 4 ChannelsPerPixel Bytes of RAM. Parallelization could be limited by environment variables