[oliverlaz]

Ticket: https://linear.app/stream/issue/RN-468

🎯 Goal

Android (GetStream/stream-chat-android#6848, GetStream/stream-chat-android#6849, GetStream/stream-chat-android#6851), iOS (GetStream/stream-chat-swift#4338) and web (GetStream/stream-chat-react#3339) added two voice-agent visualizations to their AI components. This brings them to stream-chat-react-native/ai-components (and stream-chat-expo/ai-components) with the same motion, so the platforms look and move alike, and shows them in a new voice mode in ExpoChatGPTSample.

🛠 Implementation details

Public API (from the ai-components entry point):

• VoiceWave: a single string carrying a wave packet that erupts with speech, its colors flowing along it.
• VoiceGlow: a soft globe whose color masses orbit inside it and swell with speech.
• useDictationAudioLevel(): the microphone's live level while dictation listens, as a Reanimated SharedValue<number | null>.
• Types VoiceAgentState, VoiceVisualizationProps, VoiceWaveProps, VoiceGlowProps, DictationLevel; useDictationState and useDictationTranscript are now exported too.
• An optional voiceVisualization theme key: colors (default palette, the accent color) and backdropColor (light or dark look; the system color scheme when unset).

Both components take state (idle | connecting | listening | thinking | speaking), colors (one to five colors), expressivity (0 to 2) and audioLevel (a SharedValue, read once a frame on the UI thread; null animates speech on its own). Other View props go to the root.

How it's built:

• One motion driver, ported from Android's VoiceVisualizationDriver (per-state motion table, frame-rate-independent easing, reduced motion at 0.07x, tap bounce, HSV palette, and the Steady/Smooth level responses for glow and wave). It's plain objects and worklets, advanced in a Reanimated useFrameCallback, so the motion runs on the UI thread and keeps going while JS is busy streaming a reply.
• VoiceGlow is a Skia RuntimeEffect: Android's AGSL shader is SkSL, so it's ported nearly verbatim and runs on every Android version. Every glow shares one compiled effect. There is no Canvas/CSS fallback and no rendering prop.
• VoiceWave is a Skia path with a 13-stop gradient, as Android draws it; its speech envelope uses the shader's float32 simplex noise (as on iOS and the web).
• Drawn at most 2x for the glow and 3x for the wave, as on iOS.
• Touches are only observed on a hit area the size of the shape, so a parent Pressable still fires and touches beside the shape pass through. Hover ripples with a mouse or pen.
• Accessibility follows native: "Voice agent" as the label and the state as the value; Reduce Motion is honoured.
• Where native and web differ, this follows native (e.g. no web-only wave smoothing).

Native dictation:

• New AIDictationOnLevel event (no codegen spec change; it goes through the existing NativeEventEmitter). Android maps the recognizer's onRmsChanged; iOS measures the audio tap in dBFS, throttled to ~30 Hz, tuned on an iPhone for .measurement mode (-60 to -35 dBFS).
• Fix: iOS dictation now remembers the AVAudioSession configuration it found and restores it when dictation ends. It used to leave the session in the .record category, which muted all playback in the app after the first dictation and kept other apps' audio ducked.

Sample: ExpoChatGPTSample gets a full-screen voice mode (VoiceGlow for the agent, VoiceWave on the microphone level). It listens with dictation, sends the transcript, follows the agent's AI state, reads the reply aloud with expo-speech and listens again. The send path moved into a shared useSendAIMessage, and the demo backend is now called with platform: 'openai', as in the React example (its Anthropic key currently fails /summarize).

Entry isolation: everything lives under src/ai-components; validate-entry-isolation passes.

🎨 UI Changes

Voice mode in ExpoChatGPTSample, on a physical iPhone 15 Pro Max and the iOS simulator. Screenshots and a recording to be added.

🧪 Testing

• Unit tests: Android's 22 driver tests and the bounce tests ported; palette, input clamping, simplex noise against GPU readback values, hit areas, wave geometry, rendering, accessibility label and value, theme fallbacks, reduced motion, and the dictation level hook. yarn test:unit, yarn lint, yarn typecheck and yarn build pass.
• The SkSL shader compiles and renders under CanvasKit, matching Android's Paparazzi snapshot.
• iOS simulator: both visualizations render and animate, and the voice-mode loop runs (listening, thinking, speaking, back to listening). Speech recognition itself fails on iOS 27 simulators ("Failed to initialize recognizer"), so transcription was faked there.
• Physical iPhone: dictation, the mic level driving the wave, the spoken reply and the full voice loop work.
• Not yet tested: Android on a device (the Android level mapping is untuned), SampleApp.

☑️ Checklist

• I have signed the Stream CLA (required)
• PR targets the develop branch
• Documentation is updated (AI components README; the React Native v9 AI integration docs are in a separate getstream.io PR)
• New code is tested in main example apps, including all possible scenarios
  • SampleApp iOS and Android
  • Expo iOS and Android (iOS done, Android pending)

[Stream-SDK-Bot]

SDK Size

title	develop	branch	diff	status
js_bundle_size	2036 KB	2036 KB	0 B	🟢
ai_components_size	130 KB	163 KB	+33403 B	🔴