KWS for tensorflow lite micro #60

Activity

1. Burton2000 commented on Sep 26, 2024

   

   Burton2000

   on Sep 26, 2024

   Last edited by Burton2000

   Contributor

   More actions

   You would have to do the feature extraction within your pre-processing code, you can see an example of how to do this here: https://github.com/ARM-software/ML-examples/tree/506c941bebdeb55aedc1b8cc53f27c482cf67ec8/tflu-kws-cortex-m/kws_cortex_m/Source/MFCC or here: https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ml-embedded-evaluation-kit/+/refs/heads/main/source/application/api/use_case/kws/src/KwsProcessing.cc
2. ctwillson commented on Sep 27, 2024

   

   ctwillson

   on Sep 27, 2024

   Author

   More actions

   You would have to do the feature extraction within your pre-processing code, you can see an example of how to do this here: https://github.com/ARM-software/ML-examples/tree/506c941bebdeb55aedc1b8cc53f27c482cf67ec8/tflu-kws-cortex-m/kws_cortex_m/Source/MFCC or here: https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ml-embedded-evaluation-kit/+/refs/heads/main/source/application/api/use_case/kws/src/KwsProcessing.cc

   Thanks for your reply. BTW，Streaming processing is very important for keyword spotting,does ML-Zoo support it? As your know,for microcontrollers,the lantency is very important
3. Burton2000 commented on Sep 27, 2024

   

   Burton2000

   on Sep 27, 2024

   Contributor

   More actions

   The ML-Zoo repository is only for providing ML models for people to use. It isn't focused on showing complete end to end embedded applications.

   The links I provided above show how to use these KWS model in a streaming audio use case so should be helpful for you.
4. ctwillson commented on Sep 28, 2024

   

   ctwillson

   on Sep 28, 2024

   Last edited by ctwillson

   Author

   More actions

   The ML-Zoo repository is only for providing ML models for people to use. It isn't focused on showing complete end to end embedded applications.

   The links I provided above show how to use these KWS model in a streaming audio use case so should be helpful for you.

   For model side on embedded applications,actually,we don't need send 2s(or 1s) audio frame to predict because the audio stream is sequential，we just need like 10ms and then combine this.Sorry for my poor English,but you can read this paper https://arxiv.org/abs/2005.06720. I have no idea about how to do that.Do u have any suggestion?
5. Burton2000 commented on Sep 30, 2024

   

   Burton2000

   on Sep 30, 2024

   Contributor

   More actions

   Thanks for sending the paper link, I understand your question now.

   The answer is no we do not provide any implementation of this type of streaming. We only provide a sample of doing what happen in fig 1.a, e.g. get enough audio for 1 inference, do an inference, get some more audio, do another whole inference, repeat