examples: iOS live streaming ASR demo - #35
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new examples/ios/ SwiftUI demo app that performs live, on-device streaming ASR using parakeet.cpp’s streaming C-API, along with a script to build a static Parakeet.xcframework and an XcodeGen project spec to generate the Xcode project.
Changes:
- Introduces an iOS SwiftUI demo app that captures mic audio with
AVAudioEngine, resamples to 16 kHz mono, and feedsparakeet_capi_stream_*for streaming transcription. - Adds a build script to produce
vendor/Parakeet.xcframework(device + simulator) and vendor the C-API header for Swift bridging. - Adds XcodeGen
project.yml, README build instructions, and gitignores for generated/bundled artifacts.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| examples/ios/scripts/build_xcframework.sh | Builds a static xcframework (device + simulator) and vendors parakeet_capi.h for Swift. |
| examples/ios/README.md | Documents setup/build steps and demo constraints for the iOS example. |
| examples/ios/project.yml | XcodeGen spec defining the app target, framework dependency, and build settings. |
| examples/ios/ParakeetDemo/Sources/TranscriberModel.swift | Main UI model: permission, model/session lifecycle, feeding audio into streaming decode, publishing transcript/waveform state. |
| examples/ios/ParakeetDemo/Sources/ParakeetSession.swift | Thin Swift wrapper over the streaming C-API with serialized access and memory management. |
| examples/ios/ParakeetDemo/Sources/ParakeetDemoApp.swift | SwiftUI app entry point. |
| examples/ios/ParakeetDemo/Sources/Parakeet-Bridging-Header.h | Exposes parakeet_capi.h to Swift via bridging header. |
| examples/ios/ParakeetDemo/Sources/ContentView.swift | SwiftUI view for waveform, start/stop, language picker, and streaming transcript display. |
| examples/ios/ParakeetDemo/Sources/AudioStreamer.swift | Captures mic audio and resamples to 16 kHz mono float blocks with an RMS level for visualization. |
| examples/ios/.gitignore | Ignores generated Xcode project, built artifacts, and bundled GGUF model resource. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
It's a genuinely nice addition and the streaming integration looks well done. But my main hesitation is maintenance. Neither I nor @richiejp have an Apple/macOS setup to build, run, or test an iOS target, so once this lands in the tree we'd be carrying code we can't actually exercise. That means we couldn't catch regressions, validate changes against new Xcode/SwiftUI/AVAudioEngine versions, or confidently review future updates to it, which isn't fair to either the project or to users who'd assume it's supported. So rather than merging it into the core repo as-is, I'd love for this to live as a community-maintained example. If someone with an Apple toolchain is willing to step up and own it, keep it building, respond to issues, and update it as the C-API evolves, I'm very happy to support that, whether that's here with a clear "community-maintained, untested by core" note, or as a linked external/companion repo. Really appreciate the effort, I just want to be upfront that we can't commit to maintaining something we have no way to test. |
|
Makes complete sense, I'd rather you not carry something you have no way to test. I'll keep the iOS demo as a personal example rather than ask Core to maintain it. Happy to have it linked from the README as "community-maintained, untested by core" whenever useful. Thanks for the thoughtful review and for the library! |
I'd be happy to link to in the README and have an apposite section for community projects, feel free to open up a PR once up, and I'll review it 👍 |
Adds
examples/ios/, a SwiftUI demo that does live, on-device streaming ASR with parakeet.cpp: tap the mic, talk, watch the transcript stream in. Waveform on top, start/stop button in the middle, live transcript at the bottom, with a language picker.It links
libparakeetstatically (Metal, embedded shader library) and drives the streaming C-API (parakeet_capi_stream_*) over mic audio captured withAVAudioEngineand resampled to 16 kHz mono.Layout
project.yml— XcodeGen spec (source of truth;.xcodeprojis generated/gitignored)scripts/build_xcframework.sh— cross-compiles libparakeet + ggml for device + simulator intoParakeet.xcframeworkParakeetDemo/Sources/— SwiftUI app,AVAudioEnginecapture, and a thin Swift wrapper over the streaming C-APIVerified
ios-arm64+ios-arm64_x86_64-simulatornemotron-3.5-asr-streaming-0.6b)Caveats
stream_begin.Happy to gate this behind discussion if an in-tree iOS example isn't wanted.