WIP: WebAudio Worklet shared memory

leroycep commented

2022-09-16 04:36:00 +00:00

(Migrated from github.com)

This PR will make the WebAudio backend use a AudioWorkletNode that shares memory with main thread.

serve files with correct headers to fulfill security requirements
require atomics and bulk_memory features for wasm32 target
~~set LibExeObjStep.import_memory = true for wasm32 targets~~
set LibExeObjStep.shared_memory = true for wasm32 targets
~~get the max_memory of the WASM module before instantiating it, as the spec requires max_memory be specified for shared memory~~
- ~~could write it out to the template in the build step~~
- ~~might be able to retrieve the information using WebAssembly.Module.imports, need to do more research~~
- turns out that WASM with shared memory exported will be instantiated properly
write an AudioWorklet script that will instantiate the module in an AudioWorkletProcessor
initialize the AudioContext and create an AudioWorkletNode and before calling wasmInit to avoid issues with initializing memory twice
- this means writing some more javascript, doesn't seem wise to use sysjs when the wasm memory will be reinitialized
- could use sysjs if the WASM module was split into two parts, but this would take more effort at the moment
  - would need to make a build script add a compile option to zig

That's all stuff that would need to be done before doing the work to match the sysaudio interface. This repository demos a webassembly module that does most of these steps for a standalone module.

By selecting this checkbox, I agree to license my contributions to this project under the license(s) described in the LICENSE file, and I have the right to do so or have received permission to do so by an employer or client I am producing work for whom has this right.

This PR will make the WebAudio backend use a AudioWorkletNode that shares memory with main thread. - [x] [serve files with correct headers to fulfill security requirements](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer#security_requirements) - [ ] require `atomics` and `bulk_memory` features for `wasm32` target - [x] ~~set `LibExeObjStep.import_memory = true` for `wasm32` targets~~ - [x] set `LibExeObjStep.shared_memory = true` for `wasm32` targets - [x] ~~get the `max_memory` of the WASM module before instantiating it, as [the spec requires `max_memory` be specified for shared memory](https://github.com/WebAssembly/threads/blob/master/proposals/threads/Overview.md#spec-changes)~~ - ~~could write it out to the template in the build step~~ - ~~might be able to retrieve the information using `WebAssembly.Module.imports`, need to do more research~~ - turns out that WASM with shared memory exported will be instantiated properly - [ ] write an AudioWorklet script that will instantiate the module in an `AudioWorkletProcessor` - [ ] initialize the `AudioContext` and create an `AudioWorkletNode` and before calling `wasmInit` to avoid issues with initializing memory twice - this means writing some more javascript, doesn't seem wise to use sysjs when the wasm memory will be reinitialized - [could use sysjs if the WASM module was split into two parts](https://github.com/WebAssembly/threads/blob/master/proposals/threads/Overview.md#initializing-memory-only-once), but this would take more effort at the moment - would need to make a build script add a compile option to zig That's all stuff that would need to be done before doing the work to match the sysaudio interface. This repository [demos a webassembly module that does most of these steps for a standalone module](https://github.com/leroycep/webaudio-shared-memory/). ---- - [x] By selecting this checkbox, I agree to license my contributions to this project under the license(s) described in the LICENSE file, and I have the right to do so or have received permission to do so by an employer or client I am producing work for whom has this right.

emidoots commented

2022-09-16 23:56:51 +00:00

(Migrated from github.com)

FYI I just merged https://github.com/hexops/mach/pull/537 which may give you some conflicts, should be easy to fix though - let me know if not

leroycep commented

2022-09-17 00:20:36 +00:00

(Migrated from github.com)

Should be good, I based this branch off of the other one.

emidoots commented

2022-09-17 06:36:52 +00:00

(Migrated from github.com)

Hm, github says there are conflicts:

Maybe it just needs a git rebase origin/main ?

Hm, github says there are conflicts: <img width="867" alt="image" src="https://user-images.githubusercontent.com/3173176/190844021-14376bf4-4cf9-45a5-8006-092216eafd9b.png"> Maybe it just needs a `git rebase origin/main` ?

leroycep commented

2022-09-17 18:08:52 +00:00

(Migrated from github.com)

I just had to rebase, not exactly sure what I did to make it not recognize that. Really been struggling with git on these branch 😅

I just had to rebase, not exactly sure what I did to make it not recognize that. Really been struggling with git on these branch :sweat_smile:

leroycep commented

2022-09-26 03:58:57 +00:00

(Migrated from github.com)

Alright, an update on this.

LLVM protects from double initialization of memory

We don't have to implement anything for this -- LLVM sets a global variable to protect from double initialization already. This means I can switch some of the code over to using sysjs and stop worrying about it.

`$__stack_pointer` is a instance global

This means that we don't have to worry about the $__stack_pointer conflicting between threads, since each thread will have it's own instance.

However this is not enough since the region of memory that is the stack is still shared between threads.

We can tell LLVM to export `__stack_pointer`

This allows us to modify the stack pointer from JavaScript like so:

instance.exports.__stack_pointer = 1337;

We can use this to write an API where we allocate the stack for a new frame at runtime.

This does require one more feature to be enabled for the WASM "CPU", mutable_globals. And we have to manually specify the symbols we want to export. But I dont't think those will be dealbreakers.

I think I've gathered all the pieces of the puzzle at this point, the rest is implementation. I'll try to find some time to implement it this week.

Alright, an update on this. ## LLVM protects from double initialization of memory We don't have to implement anything for this -- LLVM sets a global variable to protect from double initialization already. This means I can switch some of the code over to using `sysjs` and stop worrying about it. - [LLVM - [WebAssembly] Initialize memory in start function](https://reviews.llvm.org/D65783?id=218569) - [Thanks to mike#6589 on the zig discord for finding the above link](https://discordapp.com/channels/605571803288698900/1023770116992811099/1023784565296341142) ## `$__stack_pointer` is a instance global This means that we don't have to worry about the `$__stack_pointer` conflicting between threads, since each thread will have it's own instance. However this is not enough since the _region of memory_ that is _the stack_ is still shared between threads. ## We can tell LLVM to export `__stack_pointer` This allows us to modify the stack pointer from JavaScript like so: ```js instance.exports.__stack_pointer = 1337; ``` We can use this to write an API where we allocate the stack for a new frame at runtime. This does require one more feature to be enabled for the WASM "CPU", `mutable_globals`. And we have to manually specify the symbols we want to export. But I dont't think those will be dealbreakers. I think I've gathered all the pieces of the puzzle at this point, the rest is implementation. I'll try to find some time to implement it this week.

leroycep commented

2022-09-27 02:15:08 +00:00

(Migrated from github.com)

Latest commit is generating sound as it should! There is still a bunch of cleanup to do, but it works

emidoots commented

2022-09-30 19:28:28 +00:00

(Migrated from github.com)

Sorry I haven't had a chance to dive into this yet. I am very excited about this work, but will be travelling the next few weeks so may not have a chance to look at this in depth until after. Hope that's okay!

leroycep commented

2022-09-30 21:05:16 +00:00

(Migrated from github.com)

Yep, that's okay. There's still some work to be done cleaning it up. I'll mark it ready for review once that's done.

emidoots commented

2023-06-25 08:18:01 +00:00

(Migrated from github.com)

https://github.com/hexops/mach/issues/823

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Rows
Columns

WIP: WebAudio Worklet shared memory #543

LLVM protects from double initialization of memory

$__stack_pointer is a instance global

We can tell LLVM to export __stack_pointer

Pull request closed

`$__stack_pointer` is a instance global

We can tell LLVM to export `__stack_pointer`