transcribe_streaming speech example does not respond after 10-30secs

[sngc1]

After a few successful streaming recognition, transcribe_streaming.py does not respond without any message, in many cases it hangs and requires to kill pyhton. Not quite clear why, but it seems that it occurs after 1-10 recognition or 10-30 secs.

Are there any way to investigate what is going on, e.g. to show logs/messages? Not clear it is because this sample code, or GCP speech service.

[bprasana]

I am facing a similar issue too. I tried chanings the DEADLINE_SECS, hoping to get better results. But it just freezes.

[puneith]

@sngc1 There has been similar issue reported #515 whose PR is in flight. Can you check in the mean time #527 if this works for you?

[jerjou]

I just pushed a not-insignificant update to the streaming sample. Would you mind trying that, and seeing if that helps things? (it's not necessarily related, but might be)

[sngc1]

@jerjou As a quick answer, the commit 5fca324 still hangs after a few recognitions (or a few 10 seconds?) and needs to kill python. (on MacOS + built-in Microphone, Japanese recognition)

@puneith Thanks, the analysis in #515 makes sense, I will investigate different RATE/CHUNK settings.

BTW, I am now working on Japanese recognition"language_code='ja-JP', which looks too slow to get result with 'is_final: true', comparing to "language_code='en-US' (much faster than JP recognition & seems works fine). Maybe another issue, but is it relevant?

[jerjou]

Re: japanese vs english

Hm... I took a clean repo, modified the language to be ja-JP, and ran it, and it seemed (subjectively) to perform comparably to english recognition, although I admit I only know a handful of short japanese phrases, so perhaps that skews my test. After setting interim_results to True, I was also able to see interim transcriptions at about the same rate I'm used to seeing them in english..

Hm.. some things for you to try:

• Try using an external microphone - even the microphone on a webcam will usually capture better audio than the built-in microphone
• Have you taken a look at the best practices page? See if any of that is relevant to your setup.

Re: hanging after a few recognitions

• How much time has elapsed since you started your request? The API currently limits streaming recognition to 1 minute, so it will stop transcribing after a minute (grpc streaming example crashes or hangs due to deadline seconds #517).
• Another possibility is - I see you've modified the sample code to add interim_results=True. Did you also set single_utterance=True? In that case it will stop transcribing when it detects the end of an utterance.

Miscellany

I added SIGINT handling to the sample code, so you should be able to use Ctrl-C to stop the script instead of having to kill it (though I've been having a bit of trouble handling if an unexpected exception is thrown..)

[sngc1]

By using an external (USB) microphone, the sample works better, while still stop responding sometimes.
Followed the best practices; checked that my microphone works fine under the Pyaudio settings in transcribe_streaming.py (tested in a simple Pyaudio script without GCP speech API); confirmed that single_utterance=True is not set.

One thing I have found is, when language_code=en-US it works fine, while in ja-JP, it sometimes stops recognition (response never returned) after a couple of recognitions. When it happens is not clear.
DEADLINE_SECS does not affect this issue, the transcribe always finishes after 60 seconds with Client GRPC deadline too short message & I believe it is proper behavior. The issue above happens before this message.
Possibly CGP side issue in ja-JP???

BTW is audio_stream thread-safe? Many times I still cannot shutdown the transcribe: I suspect the audio_stream object shared by fill_buffer_thread and main thread prevents shutting down when called audio_stream.stop_stream() while buff.put(audio_stream.read(chunk)) still ongoing?

[thecloudist]

I have working code now using the transcribe_streaming_thread.py version which was shared.
Should I be fetching and syncing with the latest repo updates to python-docs-samples?
That python file does not exist and I also have made some changes to 'listen_print_loop'. I now call mine listen_transcribe_loop.

I have not tried the OOB version of transcribe_streaming.py again to see if I cleared something else after you provided the threaded version.

By the way, it was a little tricky to convert transcription strings into dictionaries which I pass to my DriveTo method that moves the robo-car according to voice commands. But it does work.

The main puzzle to solve now is how to expand my overall control code without having to put everything inside the listen_transcribe_loop.

How do I call the transcribe loop without getting stuck there?

Suggestions?

[puneith]

@thecloudist I would recommend you get the updated sample code since it has final fixes.

re: How do I call the transcribe loop without getting stuck there?
If I am assuming correctly on what you mean here, you need to either put the transcribed response in a queue or create a stream where the output is written and read from it.

[thecloudist]

The queue idea is good but the way the transcribe_print_loop() was written, I cannot exit from that method until a terminal condition is reached such as 'quit' etc.

I am going to have to break that method apart so I can just ask for transcription when my main code loop is ready to ask for it rather than having it looping all the time. The app is kind of an asynchronous command and control app for a robot car.

Thanks!

[sngc1]

Regarding the recognition hanging issue, after dozens of trials, language_code=ja-JP recognition is still unstable: on average the recognition hangs after 3 utterances. Confirmed it is BEFORE the deadline 60 secs.

Due to running out of time to investigate, my workaround is to use single_utterance=True; continuously terminate & restart recognition for each utterance.

[jerjou]

Hello!

Sorry for the long delay - the backend folks have confirmed that this is probably an issue on their end that they're actively working on. If you wouldn't mind providing a link to a sample of audio that stops transcribing after awhile, it'd help them in fixing the issue.

Thanks!

[theacodes]

(Closing due to inactivity, feel free to comment and we'll re-open)