💡 Full-stack on-device AI inference via ExecuTorch for offline-capable inspections

[github-actions[bot]]

Summary

Integrate React Native ExecuTorch to run Whisper STT, photo classification (varroa/brood patterns), and a fine-tuned small LLM (SmolLM 2 or Qwen 3) for recommendations entirely on-device. This enables fully offline AI-powered inspections with zero cloud dependency, complete data privacy, and sub-second inference — a capability no competitor currently offers end-to-end.

Market Signal

• HiveSense already ships on-device Whisper (142MB offline model) for offline voice transcription — validating the on-device approach
• Beentry, APiLOG, HiveMasterPro, Hive Pal, BeeKeeperVoice all offer voice AI, but cloud-dependent
• React Native ExecuTorch reached production maturity in 2026: sub-20ms inference on modern devices, supports Whisper/CLIP/Llama 3.2/Qwen 3, with Expo config plugin
• HiveLog AI leads in photo-based varroa detection but requires cloud upload
• 60%+ of apiaries have unreliable cell coverage during inspections
• Smart beekeeping market growing 20% CAGR to $1.8B by 2032

Sources: React Native ExecuTorch docs, HiveSense offline features, Best Beekeeping Apps 2026

User Signal

Existing idea #346 proposes on-device voice transcription specifically. This proposal extends far beyond STT to the full inference pipeline: voice → structured data → photo analysis → AI recommendations, all on-device. Core differentiator: zero-tap inspections work fully offline with intelligence, not just data capture.

Technical Opportunity

ExecuTorch has a React Native library (Software Mansion) with Expo config plugin support. Key capabilities:

• Whisper for ASR (1% WER on clean speech, 55% fewer errors on noisy audio)
• CLIP for image embedding and classification
• Small LLMs (SmolLM 2, Qwen 3) for text generation and recommendation reasoning
• iOS delegates to CoreML/Metal; Android to QNN/XNNPACK
• 50KB base runtime, models 142MB–1.5GB depending on capability tier

Broodly's architecture already defines Vertex AI for cloud inference — on-device inference becomes a parallel execution path with graceful cloud fallback. The phased approach: Whisper first, photo ML second, recommendation LLM third.

Assessment

Dimension	Score	Rationale
Feasibility	med	ExecuTorch is production-ready but new to Expo ecosystem. HiveSense validates the on-device Whisper approach. Phased rollout mitigates risk.
Impact	high	Only app with full on-device intelligence pipeline. Privacy-first, works-everywhere architecture is an enduring competitive moat.
Urgency	high	Voice-first inspection is becoming table stakes (6+ competitors). On-device execution is the next differentiator wave.

Adversarial Review

Strongest objection: ExecuTorch is relatively new technology. On-device models are inherently less capable than cloud models (GPT-4 vs SmolLM 2). Engineering effort is significant and may distract from core MVP feature delivery.

Rebuttal: Whisper Large v3 achieves 1% WER on-device — competitive with cloud APIs. Phased rollout delivers standalone value at each phase: Phase 1 ships on-device Whisper (proven, HiveSense validates), Phase 2 adds photo classification (TFLite models are mature), Phase 3 adds recommendation LLM. The privacy-first, works-everywhere architecture is a moat, not a distraction.

Suggested Next Step

Spike: Integrate react-native-executorch with Expo config plugin, benchmark Whisper inference latency on iPhone 12+ and Pixel 6+, validate model download and storage UX. Document minimum device requirements and model size tradeoffs.

[prashantweb3task-collab]

This is a really interesting direction. Running the entire AI pipeline on-device instead of relying on the cloud could be a huge advantage, especially for field work where internet connectivity is unreliable. The privacy benefits are another strong selling point since sensitive inspection data never has to leave the device.

I also like the phased rollout approach. Starting with offline speech recognition before expanding to image analysis and on-device LLMs feels much more practical than trying to ship everything at once.
One thing I'd be curious about is how model size and battery usage will affect the overall user experience on mid-range Android devices. It might also be worth giving users the option to download only the AI models they actually need to reduce storage requirements.
Overall, this seems like a solid long-term architecture. If the performance benchmarks are as good as expected, it could set a new standard for offline AI-powered mobile applications.

[github-actions[bot]]

Weekly Update — 2026-07-17

What Changed

ExecuTorch v0.8 shipped with significant expansions. New capabilities include instance segmentation, OCR, and vision-language models. QNN delegate stability on Snapdragon 8 Elite has improved — this is the hardware acceleration path for Android flagship devices.

React Native ExecuTorch library now provides production-ready hooks. The react-native-executorch library by Software Mansion offers a comprehensive React hook API:

• useLLM — model loading, token streaming, memory management for on-device chat/reasoning
• useWhisper — on-device speech-to-text (Whisper via ExecuTorch)
• useImageClassification, useObjectDetection, useOCR — computer vision tasks

This is directly usable from Broodly's Expo + React Native stack, providing a single integration point for offline voice transcription, image classification (brood pattern, pest detection), and potentially on-device reasoning.

Competitive validation. HiveSense has shipped on-device Whisper for voice transcription. On-device AI is no longer experimental in the beekeeping app category — it's a shipping feature at a direct competitor.

Gemini Nano hybrid inference. Google's April 2026 announcement enables Android apps to dynamically switch between Gemini Nano (on-device) and cloud-hosted Gemini models. This is relevant for Broodly's Vertex AI integration: start with cloud inference, fall back to on-device when offline.

Sources: React Native ExecuTorch, On-Device AI 2026, React Native On-Device AI Tutorial, Gemini Nano Hybrid Inference

Updated Assessment

Dimension	Previous	Current	Delta
Feasibility	med	high	↑ react-native-executorch provides turnkey hooks; ExecuTorch v0.8 production-stable
Impact	high	high	→ unchanged — offline AI remains a major differentiator
Urgency	med	high	↑ competitor has shipped on-device voice; Gemini Nano hybrid path available

Recommendation

Advance. The React Native ExecuTorch integration path is now clear and practical. Recommend a phased approach: (1) integrate useWhisper for offline voice transcription as part of Epic 11, (2) evaluate useImageClassification for on-device brood pattern pre-screening to reduce cloud API calls, (3) explore Gemini Nano hybrid inference for Android users as a cloud-fallback optimization. The technology risk has dropped significantly since this Discussion was created.

[github-actions[bot]]

Weekly Update

What Changed

New technical path: Gemini Nano as alternative to ExecuTorch. Google's Gemini Nano is now running on over 140 million Android devices with a production-ready API (AICore). Callstack has announced React Native AI integration for on-device inference, making Gemini Nano directly accessible from Broodly's Expo/RN stack without custom model compilation. Key advantage: 50-200ms first-token latency, multimodal support (text + image), and no per-request cloud cost.

New competitive urgency: HiveSense ships on-device Whisper. HiveSense, a new beekeeping app competitor identified in this week's market scan, has shipped on-device voice transcription using Whisper. They position as "offline-first" with "every feature works without internet, including voice-to-text processing." This validates the market demand for on-device AI in beekeeping and increases urgency for Broodly to ship equivalent capabilities.

Android vs. iOS path divergence. Gemini Nano is Android-only via AICore. For iOS, the equivalent path is Core ML with Apple's on-device models. A practical implementation guide (Wednesday.is, 2026) demonstrates bridging both runtimes from React Native via native modules — one JS API, two native backends.

Updated Assessment

Dimension	Previous	Current	Delta
Feasibility	med	med	→ (Gemini Nano is more turnkey than ExecuTorch, but dual-platform bridging adds complexity)
Impact	high	high	→
Urgency	med	high	↑ (HiveSense competitive pressure; Gemini Nano maturity reduces "wait and see" argument)

Recommendation

Advance. Evaluate Gemini Nano alongside ExecuTorch as the on-device inference runtime. Gemini Nano offers a faster path to shipping (no custom model compilation, production-ready API) at the cost of Android-first implementation. Suggested next step: build a proof-of-concept native module that runs a simple text classification task via AICore on Android and Core ML on iOS, measuring latency and accuracy on target beekeeping vocabulary.

[github-actions[bot]]

Weekly Update

What Changed

React Native ExecuTorch v0.8 — production-ready milestone:
React Native ExecuTorch has reached v0.8 with significant maturity improvements:

• Expanded computer vision: YOLO, RF-DETR, MobileNet, Segment Anything (SAM), instance segmentation, OCR, vision-language models
• Sub-20ms latency for production CV models on modern phones (benchmarked by AlephZero Labs, 2026)
• Speech: Whisper (STT) + Kokoro (TTS) included out of the box
• Embeddings: On-device vector embeddings for semantic search / local RAG
• Hardware acceleration: XNNPACK (CPU), Core ML & MLX (Apple Silicon), Vulkan (Android GPU)
• Task-specific React hooks: use<Task>() with automatic model caching and lifecycle management
• On-demand model downloads: Models don't need to ship in the app bundle, addressing the app size concern

Market validation — phone-based bee analysis:

• BeeScanning: 17,000+ users worldwide, found 236,000+ varroa mites in 140,000 hives using phone cameras
• HiveLog AI: Photo-based varroa detection with mite count, risk percentage, and confidence score (€3-8/mo)
• BeeSentry: IoT + satellite + AI analytics, 84% varroa detection accuracy, preparing for 2026 market launch
• Tiny-LiteNet (research): 98.6% accuracy, 80ms inference, 1.2MB model, 1.48M parameters — outperforms MobileNetV2 on pest detection

Emerging architecture pattern:
A tiered approach is crystallizing in agricultural AI: lightweight CNN for real-time on-device scanning, with ambiguous cases flagged for server-side analysis via larger models when connectivity allows. This maps perfectly to Broodly's architecture — ExecuTorch for instant field feedback, Vertex AI for confident server-side analysis.

Updated Assessment

Dimension	Previous	Current	Delta
Feasibility	med	high	↑ (v0.8 is production-ready with task-specific React hooks; pre-exported YOLO models available)
Impact	high	high	→
Urgency	med	med	→ (foundational capability; enables Discussions #497, #506, #528 but isn't user-facing alone)

Recommendation

Advance — feasibility is now confirmed. The v0.8 release with task-specific React hooks and sub-20ms inference makes this production-ready for Broodly. Concrete next steps:

1. Proof of concept: Use react-native-executorch with a pre-exported YOLO model to benchmark varroa detection accuracy on sample inspection photos. Measure inference latency across iPhone 14, iPhone 16, Pixel 8, and Galaxy S24.
2. Tiered architecture design: Define the on-device → server fallback interface for the three blocked discussions (💡 Real-time brood frame scoring via on-device multimodal AI during zero-tap inspections #497 brood frame scoring, 💡 AI-powered phone-camera Varroa mite counting with closed-loop treatment decision support #506 varroa counting, 💡 Photo-AI equipment and frame condition assessment for proactive comb management #528 equipment assessment).
3. Kokoro TTS evaluation: Benchmark on-device Kokoro TTS quality and latency for the voice briefing feature (Discussion 💡 AI-generated voice briefing and debrief for zero-tap inspection lifecycle #477) as an alternative to Vertex AI TTS — eliminates cloud dependency for voice output.
4. On-device embeddings: Evaluate ExecuTorch embeddings for the conversational voice assistant (Discussion 💡 Conversational voice assistant grounded in hive history #87) — local RAG over inspection history would enable offline-capable natural language queries.