[µTVM] Print .elf statistics for a model runtime built with Zephyr by gromero · Pull Request #7449 · apache/tvm

gromero · 2021-02-11T23:55:13Z

Hi,

Could the following simple change be reviewed please?

Currently there isn't any statistics about the used resources by a model
runtime built with Zephyr, making it difficult to have any idea about, for
instance, the amount of memory taken by the operations necessary to run the
model.

Since Zephyr's SDK already exposes the statistics about various memory
regions on linking by passing '--print-memory-usage' to the linker, it's
possible to use it to have an idea about the amount of memory used by the
model and how much memory is left on the device.

That commit adds a simple method to extract the memory region information
out of the build output and then uses it to show memory usage statistics
for various memory regions when Zephyr finishes building the image to be
flashed to the target device.

Output example follows:

gromero@gromero0:~/git/tvm/tutorials/micro$ python3 ./micro_tflite.py 
File /home/gromero/.tvm_test_data/data/sine_model.tflite exists, skip.
Model Version: 3
<snip>
Including boilerplate (Zephyr base): /home/gromero/zephyrproject/zephyr/cmake/app/boilerplate.cmake
Including boilerplate (Zephyr base): /home/gromero/zephyrproject/zephyr/cmake/app/boilerplate.cmake
Including boilerplate (Zephyr base): /home/gromero/zephyrproject/zephyr/cmake/app/boilerplate.cmake
Including boilerplate (Zephyr base): /home/gromero/zephyrproject/zephyr/cmake/app/boilerplate.cmake
Including boilerplate (Zephyr base): /home/gromero/zephyrproject/zephyr/cmake/app/boilerplate.cmake
Including boilerplate (Zephyr base): /home/gromero/zephyrproject/zephyr/cmake/app/boilerplate.cmake
Including boilerplate (Zephyr base): /home/gromero/zephyrproject/zephyr/cmake/app/boilerplate.cmake
==================
Memory region         Used Size  Region Size  %age Used
           FLASH:       35748 B         1 MB      3.41%
            DTCM:          0 GB        64 KB      0.00%
            SRAM:      146160 B       256 KB     55.76%
        IDT_LIST:         232 B         2 KB     11.33%
make[2]: Leaving directory '/tmp/tmpcb9im44g/build/runtime'
Open On-Chip Debugger 0.10.0+dev-01341-g580d06d9d-dirty (2020-06-25-12:07)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
066CFF485153826687133653
Info : The selected transport took over low-level target control. The results might differ compared to plain JTAG/SWD
Info : clock speed 2000 kHz
Info : STLINK V2J25M14 (API v2) VID:PID 0483:374B
Info : Target voltage: 3.238421
Warn : Silicon bug: single stepping may enter pending exception handler!
Info : stm32f7x.cpu: hardware has 8 breakpoints, 4 watchpoints
Info : Listening on port 3333 for gdb connections
    TargetName         Type       Endian TapName            State       
--  ------------------ ---------- ------ ------------------ ------------
 0* stm32f7x.cpu       hla_target little stm32f7x.cpu       running

target halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x08001e20 msp: 0x200331a8
Info : device id = 0x10016449
Info : flash size = 1024 kbytes
auto erase enabled
wrote 65536 bytes from file /tmp/tmpcb9im44g/build/runtime/zephyr/zephyr.hex in 2.129213s (30.058 KiB/s)

target halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x08001e20 msp: 0x200331b0
verified 35748 bytes in 0.489722s (71.286 KiB/s)

shutdown command invoked
result is: [[0.4443792]]

Thanks & best regards,
Gustavo

Currently there isn't any statistics about the used resources by a model runtime built with Zephyr, making it difficult to have any idea about, for instance, the amount of memory taken by the operations necessary to run the model. Since Zephyr's SDK already exposes the statistics about various memory regions on linking by passing '--print-memory-usage' to the linker, it's possible to use it to have an idea about the amount of memory used by the model and how much memory is left on the device. That commit adds a simple method to extract the memory region information out of the build output and then uses it to show memory usage statistics for various memory regions when Zephyr finishes building the image to be flashed to the target device. Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>

gromero · 2021-02-11T23:56:03Z

@tom-gall @areusch

areusch

thanks @gromero, a couple small comments but I think this is a great change!

areusch · 2021-02-12T00:11:20Z

        return tvm.micro.MicroLibrary(build_dir, [f"lib{project_name}.a"])

+    def _print_make_statistics(self, output):
+        print("==================")


want to print this after you see "Memory region"?

sure! Now the output looks like:

INFO:tvm.micro.contrib.zephyr:Memory region Used Size Region Size %age Used INFO:tvm.micro.contrib.zephyr:--------------------- ---------- ------------ --------- INFO:tvm.micro.contrib.zephyr: FLASH: 35748 B 1 MB 3.41% INFO:tvm.micro.contrib.zephyr: DTCM: 0 GB 64 KB 0.00% INFO:tvm.micro.contrib.zephyr: SRAM: 146160 B 256 KB 55.76% INFO:tvm.micro.contrib.zephyr: IDT_LIST: 232 B 2 KB 11.33%

areusch · 2021-02-12T00:11:49Z

+        print("==================")
+        output = output.splitlines()
+        lines = iter(output)
+        for line in lines:


could you catch StopIteration somewhere so that in the event the output looks weird, this doesn't break the compilation flow?

areusch · 2021-02-12T00:12:27Z

+        for line in lines:
+            if line.startswith("Memory region"):
+                # print statistics header
+                print(line)


it might be better to use _LOG.info() rather than print, so it could be suppressed if needed

done. @areusch Thanks a lot for the review.

- Catch StopIteration in case of a weird output or no additional lines after the last memory region - Use of _LOG.info() instead of plain print() for better control over the output by the main script - Set log level in micro_tflite.py script as an example on how to get the new memory usage statistics and also because currently that's the main script used to test microTVM + Zephyr's SDK - Improve statistics header Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>

It seems build system is using Python < 3.7, so 'text' argument is not present as an alias for 'universal_newlines'. To satisfy it use old 'universal_newlines' argument which is available prior to Python 3.7.

Avoid exception anti-pattern when catching StopIteration

gromero · 2021-02-15T11:01:47Z

@tmoreau89 Hi. Could you please merge it?

tmoreau89

LGTM

gromero · 2021-02-15T18:22:58Z

@tmoreau89 thanks for the review and for merging it!

…pache#7449) * [µTVM] Print .elf statistics for a model runtime built with Zephyr Currently there isn't any statistics about the used resources by a model runtime built with Zephyr, making it difficult to have any idea about, for instance, the amount of memory taken by the operations necessary to run the model. Since Zephyr's SDK already exposes the statistics about various memory regions on linking by passing '--print-memory-usage' to the linker, it's possible to use it to have an idea about the amount of memory used by the model and how much memory is left on the device. That commit adds a simple method to extract the memory region information out of the build output and then uses it to show memory usage statistics for various memory regions when Zephyr finishes building the image to be flashed to the target device. Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org> * v2: Fixes accordingly to Andrew review - Catch StopIteration in case of a weird output or no additional lines after the last memory region - Use of _LOG.info() instead of plain print() for better control over the output by the main script - Set log level in micro_tflite.py script as an example on how to get the new memory usage statistics and also because currently that's the main script used to test microTVM + Zephyr's SDK - Improve statistics header Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org> * Fix build It seems build system is using Python < 3.7, so 'text' argument is not present as an alias for 'universal_newlines'. To satisfy it use old 'universal_newlines' argument which is available prior to Python 3.7. * Fix build Avoid exception anti-pattern when catching StopIteration * Retrigger CI

areusch requested changes Feb 12, 2021

View reviewed changes

gromero added 3 commits February 12, 2021 17:21

Fix build

387a9eb

It seems build system is using Python < 3.7, so 'text' argument is not present as an alias for 'universal_newlines'. To satisfy it use old 'universal_newlines' argument which is available prior to Python 3.7.

Fix build

39aece9

Avoid exception anti-pattern when catching StopIteration

areusch approved these changes Feb 12, 2021

View reviewed changes

Retrigger CI

6ebd7fe

tmoreau89 approved these changes Feb 15, 2021

View reviewed changes

tmoreau89 merged commit 32c4402 into apache:main Feb 15, 2021

junrushao mentioned this pull request Nov 1, 2021

Apache TVM v0.8 Release Note Candidate #9416

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[µTVM] Print .elf statistics for a model runtime built with Zephyr#7449

[µTVM] Print .elf statistics for a model runtime built with Zephyr#7449
tmoreau89 merged 5 commits into
apache:mainfrom
gromero:for-rev-01

gromero commented Feb 11, 2021

Uh oh!

gromero commented Feb 11, 2021

Uh oh!

areusch left a comment

Uh oh!

areusch Feb 12, 2021

Uh oh!

gromero Feb 12, 2021

Uh oh!

areusch Feb 12, 2021

Uh oh!

gromero Feb 12, 2021

Uh oh!

areusch Feb 12, 2021

Uh oh!

gromero Feb 12, 2021 •

edited

Loading

Uh oh!

gromero commented Feb 15, 2021

Uh oh!

tmoreau89 left a comment

Uh oh!

gromero commented Feb 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

gromero commented Feb 11, 2021

Uh oh!

gromero commented Feb 11, 2021

Uh oh!

areusch left a comment

Choose a reason for hiding this comment

Uh oh!

areusch Feb 12, 2021

Choose a reason for hiding this comment

Uh oh!

gromero Feb 12, 2021

Choose a reason for hiding this comment

Uh oh!

areusch Feb 12, 2021

Choose a reason for hiding this comment

Uh oh!

gromero Feb 12, 2021

Choose a reason for hiding this comment

Uh oh!

areusch Feb 12, 2021

Choose a reason for hiding this comment

Uh oh!

gromero Feb 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gromero commented Feb 15, 2021

Uh oh!

tmoreau89 left a comment

Choose a reason for hiding this comment

Uh oh!

gromero commented Feb 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gromero Feb 12, 2021 •

edited

Loading