Skip to content

feat: add extension pump and audio callback mechanisms#790

Closed
Lebei2046 wants to merge 5 commits into
PerryTS:mainfrom
Lebei2046:feature/voice-to-text-2
Closed

feat: add extension pump and audio callback mechanisms#790
Lebei2046 wants to merge 5 commits into
PerryTS:mainfrom
Lebei2046:feature/voice-to-text-2

Conversation

@Lebei2046
Copy link
Copy Markdown
Contributor

@Lebei2046 Lebei2046 commented May 15, 2026

Summary

Added a generic extension pump registration mechanism and audio callback registration for external extensions (like Vosk speech recognition).

Changes

Feature 1: Generic Extension Pump Registration

crates/perry-runtime/src/lib.rs

  • Added ext_pump module with:

    • js_register_ext_pump(f) : Extensions call this to register their process_pending function
    • js_run_ext_pump() : Called by platform-specific timer pumps to execute registered extension work
      crates/perry-ui-gtk4/src/app.rs
  • Added declaration and call to js_run_ext_pump() in the 8ms timer pump

Feature 2: Audio Callback Registration

crates/perry-runtime/src/lib.rs

  • Added js_register_audio_callback(f) : Register callback for real-time audio data

  • Added js_unregister_audio_callback() : Unregister audio callback

  • Added js_invoke_audio_callback(data) : Invoke registered callback with audio data
    crates/perry-ui-gtk4/src/audio.rs

  • Implemented audio capture from PulseAudio

  • Integrated callback invocation when audio data is captured

  • Supports 48kHz mono audio with configurable buffer size

Related issue

Test plan

The extension pump and audio callback are used by the Vosk speech recognition extension:

  • Real-time audio capture and processing
  • Text updates during speech recognition without UI hanging
  • Proper thread-safe callback handling

Screenshots / output

Check chatroom-perry for functionalities.

Checklist

  • I have NOT bumped the workspace version or edited CLAUDE.md / CHANGELOG.md (maintainer handles these at merge)
  • My commits follow the loose feat: / fix: / docs: / chore: prefix convention used in the log
  • I've read CONTRIBUTING.md and agree to the Code of Conduct

Lebei2046 and others added 5 commits May 14, 2026 08:13
Add support for specifying library search paths in native library
target configurations via the new `libDirs` field in package.json.

Changes:
- Add `lib_dirs` field to TargetNativeConfig
- Parse `libDirs` from package.json
- Add `-L` flags for libDirs before linking
…docs (v0.5.909)

Maintainer fold-in on top of @Lebei2046's libDirs feature:

- link.rs: emit /LIBPATH:<dir> on is_windows (MSVC link.exe) instead of
  -L<dir>, mirroring the existing libs loop's MSVC/Unix split.
- resolve.rs: anchor relative libDirs entries to package_dir via
  package_dir.join(p), matching swift_sources / metal_sources. Field
  type is now Vec<PathBuf>.
- compile.rs: doc-comment on the new field.
- manifest-v1.md: row added for libDirs.
- resolve.rs: two new tests in manifest_parse_tests covering
  package_dir anchoring + empty-by-default behavior.
- Cargo.toml / Cargo.lock / CLAUDE.md / CHANGELOG.md: 0.5.908 → 0.5.909
  (renumbered to follow PerryTS#760 which took 0.5.908 mid-review).
@Lebei2046 Lebei2046 changed the title feat: add generic extension pump registration mechanism feat: add extension pump and audio callback mechanisms May 17, 2026
@proggeramlug
Copy link
Copy Markdown
Contributor

@Lebei2046 I'll attend this PR shortly, need to push through a few PRs first though!

proggeramlug added a commit that referenced this pull request May 18, 2026
…790) (#1038)

Squashes @Lebei2046's #790 onto current main as a single net diff.
Original PR's branch had been merged-rebased through multiple older
main commits whose contents are now upstream; this version preserves
only the contributor's actual code additions.

Author's content unchanged:
- crates/perry-ui-gtk4/src/{app,audio,widgets/webview}.rs — audio
  recording flow + extension pump wiring.
- crates/perry/src/commands/compile/{link,resolve}.rs — libDirs
  / extension manifest discovery (most of which is now redundant
  with main's modern code, so the resolve.rs change is reduced to
  the per-arch + prebuilt logic that's already on main).
- types/perry/system/index.d.ts — audio API signatures.
- docs/src/native-libraries/manifest-v1.md — manifest field.

Conflicts (resolved by keeping current main):
- Cargo.lock/Cargo.toml/CHANGELOG.md/CLAUDE.md — metadata, take main.
- crates/perry-ui-gtk4/Cargo.toml — keep both `base64` and
  `perry-ffi` deps.
- crates/perry-codegen/src/lower_call.rs +
  crates/perry-codegen/src/type_analysis.rs — stale `Promise.resolve`
  helper markers from the contributor's merge attempt; main's
  `is_global_constructor_expr` already covers the same case.
- crates/perry/src/commands/compile/resolve.rs — keep main's modern
  per-arch + prebuilt logic; drop the contributor's older shape.

Validation: `cargo check --release -p perry` is clean.
Credits @Lebei2046.
@proggeramlug
Copy link
Copy Markdown
Contributor

Rebased + admin-merged via #1038. Conflicts resolved by keeping main's modern per-arch / prebuilt logic and the is_global_constructor_expr Promise helper. Thanks for the audio extension work @Lebei2046! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants