[Tracking Issue] UMA: Universal Modular Accelerator Interface

[MichaelJKlaiber]

This is a global tracking issue for landing UMA. The original RFC can be found here. Please feel free to bring in more questions to the corresponding thread.

To make the improvement manageable, we will be breaking the code into several steps. The main architecture scaffolding and
data structure changes will come first, then we will upstream individual key features.

Steps

[UMA-0] RFC

• Definition of UMA-RFCv1 [RFC] UMA Universal Modular Accelerator Interface tvm-rfcs#60

[UMA-1] Implementation of core infrastructure

Implementation of UMA Backend according to UMA-RFCv1

• UMA Partitioner [UMA] UMA v1.0 #12087
• UMA Pipeline [UMA] UMA v1.0 #12087
• UMA Lower [UMA] UMA v1.0 #12087
• UMA Codegen [UMA] UMA v1.0 #12087
• Unit tests [UMA] UMA v1.0 #12087

[UMA-2] Prototype of UMA using NN accelerator

• Using UMA for deployment on UltraTrail NN accelerator (University of Tübingen, Germany)
• Using UMA for deployment on NN accelerator_2 (to be defined)

[UMA-3] Formulation of lessons learned for UMA_v1.0

• Lessons learned and future topics document based on [UMA-2]
• move OutlineCompilerFunctionsMutator out of UMA
• extend/add AOTTestRunner by project API
• annotations of UMALower._operator_strategies
• Add test case importing tflite model from .tflite file [BugFix][UMA] Fix order issue in uma_lower  #12447
• Add test case importing onnx model from .onnx file

[UMA-4] Documentation

• Vanilla Accelerator Tutorial [UMA] UMA v1.0 #12087
• Strawberry Accelerator Tutorial
• Chocolate Accelerator Tutorial

[UMA-5] Suggestion from UMAv1 PR

• Add UMA registration to tvmc
• Case where we lower some parts with UMA while handling other unsupported parts via the conventional pipeline --> Integrate MobileNet unit test case [UMA] UMA v1.0 #12087
• Tutorial as RST for gallery [UMA] UMA v1.0 #12087

Backlog

• run BERT with UMA on ARM EthosU (No commitment by community yet)

Working branch

https://github.com/MichaelJKlaiber/tvm/tree/uma

[gfvvz]

Hi, thanks for the work. Just want to know if there is a rough schedule on these features development?

[MichaelJKlaiber]

Hi @gfvvz , that's a great question. I'll try to come up with an answer asap. If you have input or feedback, feel free to reach out any time via Discord, GitHub or the TVM community forum

[MichaelJKlaiber]

@gfvvz , just added the link to our working branch, feel free to peek into (partially finished) functions.

[xinetzone]

python/tvm/relay/backend/contrib/uma/backend.py has a bug：

for name, attr in self._target_attrs:
            if attr is None:
                raise ValueError("Target attribute None is not supported.")

can use:

for attr in self._target_attrs.values():
            if attr is None:
                raise ValueError("Target attribute None is not supported.")

[areusch]

@xinetzone do you mind filing a separate issue and including some more detail (i.e. is there a stacktrace)?

[fpedd]

@xinetzone @areusch there is a PR for that already #12731

[areusch]

oh yes, thanks for the reminder @fabianpedd :)

[qzylalala]

Hi, thanks for the work. I wonder whether this issue is still active?
I sincerely hope that there are some progress in ([UMA-2] Prototype of UMA using NN accelerator) ！