gpu: build idiomatic Zig interface to WebGPU, abstract away native+browser support

emidoots commented

2021-08-04 01:12:14 +00:00

(Migrated from github.com)

This would give us a truly nice, cross-platform, seamless graphics API across browsers, mobile and desktop via Vulkan, Metal, and OpenGL as fallback.

Use the same Zig API for:

WebAssembly (browser's implementation)
Mobile (dawn or gfx-rs/wgpu-native backend, at your choosing)
Desktop (dawn or gfx-rs/wgpu-native backend, at your choosing)

This is also the only truly cross-platform graphics API being blessed by major vendors (e.g. Apple) which leads me to believe it has a real shot long-term at displacing the tried and true Vulkan+Metal+OpenGL combo. And even if it doesn't, there has been a massive undertaking to offer a unified API of that Vulkan+Metal+OpenGL combo in forming WebGPU that very few will be able to effectively beat in the general case (e.g. I think we can effectively beat Godot's, potentially Unity and Unreal's, graphics API abstractions using WebGPU.)

This would give us a truly nice, cross-platform, seamless graphics API across browsers, mobile and desktop via Vulkan, Metal, and OpenGL as fallback. Use the same Zig API for: * WebAssembly (browser's implementation) * Mobile ([dawn](https://dawn.googlesource.com/dawn) or [gfx-rs/wgpu-native](https://github.com/gfx-rs/wgpu-native) backend, at your choosing) * Desktop ([dawn](https://dawn.googlesource.com/dawn) or [gfx-rs/wgpu-native](https://github.com/gfx-rs/wgpu-native) backend, at your choosing) This is also the only truly cross-platform graphics API being blessed by major vendors (e.g. Apple) which leads me to believe it has a real shot long-term at displacing the tried and true Vulkan+Metal+OpenGL combo. And even if it doesn't, there has been a massive undertaking to offer a unified API of that Vulkan+Metal+OpenGL combo in forming WebGPU that very few will be able to effectively beat in the general case (e.g. I think we can effectively beat Godot's, potentially Unity and Unreal's, graphics API abstractions using WebGPU.)

❤️ 1

meshula commented

2021-10-25 17:11:54 +00:00

(Migrated from github.com)

For what its worth, I've brought up an easily reproducible dawn build here: https://github.com/meshula/LabSlang (it assumes you have jinja2 installed). I've been looking for a way to zigify the sample program: https://github.com/meshula/LabSlang/blob/main/example/ManualSwapChainTest.cpp and stumbled across your glfw binding which could be a nice first step, from my perspective.

emidoots commented

2021-10-25 18:14:55 +00:00

(Migrated from github.com)

@meshula Oh, a few thoughts:

I don't want to rely on CMake, jinja2, or any other build tools. Just zig build. I actually already have this working on MacOS, and other platforms should be pretty trivial/straightforward from here. I haven't published any of this yet on GitHub, though, sorry. I need to do some more work to finalize things but I'll see if I can push a branch up soon.
I have ported the Dawn triangle example to Zig, which includes writing C shims for most of the Dawn C++ utilities, swap chain logic, etc.

You can read a bit about it here: https://devlog.hexops.com/2021/mach-engine-the-future-of-graphics-with-zig#whats-next-webgpu

I'll try to get a branch up that you can look at / play with / help out with, if you're interested, sometime this week

@meshula Oh, a few thoughts: 1. I don't want to rely on CMake, jinja2, or any other build tools. Just `zig build`. I actually already have this working on MacOS, and other platforms should be pretty trivial/straightforward from here. I haven't published any of this yet on GitHub, though, sorry. I need to do some more work to finalize things but I'll see if I can push a branch up soon. 2. I have ported the Dawn triangle example to Zig, which includes writing C shims for most of the Dawn C++ utilities, swap chain logic, etc. You can read a bit about it here: https://devlog.hexops.com/2021/mach-engine-the-future-of-graphics-with-zig#whats-next-webgpu I'll try to get a branch up that you can look at / play with / help out with, if you're interested, sometime this week

emidoots commented

2021-11-04 06:58:02 +00:00

(Migrated from github.com)

Sent a PR to fix the Obj-C++ compilation issue in Zig, so we can compile Dawn using a build.zig file: https://github.com/ziglang/zig/pull/10096

Sent a PR to fix the Obj-C++ compilation issue in Zig, so we can compile Dawn using a `build.zig` file: https://github.com/ziglang/zig/pull/10096

🎉 1

meshula commented

2021-11-06 01:28:40 +00:00

(Migrated from github.com)

Sorry I missed your reply, if you post a branch, I'm definitely interested.
... which I see is here https://github.com/hexops/mach/tree/dawn

Sorry I missed your reply, if you post a branch, I'm definitely interested. ... which I see is here https://github.com/hexops/mach/tree/dawn

emidoots commented

2021-11-06 02:46:27 +00:00

(Migrated from github.com)

@meshula Cool! You indeed found the branch I'm working from, https://github.com/hexops/mach/pull/62 is the pull request which has more details on how to use it, issues, etc. I probably need about a day more before it'll be relatively easy for you to try running it

meshula commented

2021-11-06 06:10:01 +00:00

(Migrated from github.com)

I got all the way to the compiling .mm issue, as you might expect. I haven't been building zig myself, so does it make sense to wait for that to land? or is there an artifact I can download from CI somewhere?

I have a fairly large contraption right now where I am writing zig bindings over my LabSlang project, which I need for a current zig project that currently is CLI only. I'd be happy to shift my attention to mach in the short term to see if we are both pushing in the same direction. Due to the mm compilation issue, I can do other things in the meanwhile and wait a little for "relatively easy", and therefore put off learning to build zig :)

As you probably already know:

❯ zig build
error: unrecognized file extension of parameter '/Users/dp/Projects/mach/webgpu/libs/dawn/src/common/SystemUtils_mac.mm'
dawn...The following command exited with error code 1:
/Users/dp/bin/zig-macos-x86_64-0.9.0-dev.1561+5ebdc8c46/zig build-lib /Users/dp/Projects/mach/webgpu/src/dawn/dummy.zig -cflags -I/Users/dp/Projects/mach/webgpu/libs/dawn/out/Debug/gen.... etc

I got all the way to the compiling .mm issue, as you might expect. I haven't been building zig myself, so does it make sense to wait for that to land? or is there an artifact I can download from CI somewhere? I have a fairly large contraption right now where I am writing zig bindings over my LabSlang project, which I need for a current zig project that currently is CLI only. I'd be happy to shift my attention to mach in the short term to see if we are both pushing in the same direction. Due to the mm compilation issue, I can do other things in the meanwhile and wait a little for "relatively easy", and therefore put off learning to build zig :) As you probably already know: ```` ❯ zig build error: unrecognized file extension of parameter '/Users/dp/Projects/mach/webgpu/libs/dawn/src/common/SystemUtils_mac.mm' dawn...The following command exited with error code 1: /Users/dp/bin/zig-macos-x86_64-0.9.0-dev.1561+5ebdc8c46/zig build-lib /Users/dp/Projects/mach/webgpu/src/dawn/dummy.zig -cflags -I/Users/dp/Projects/mach/webgpu/libs/dawn/out/Debug/gen.... etc ````

meshula commented

2021-11-06 06:14:24 +00:00

(Migrated from github.com)

The only part of the build that required a bit of research was the installation of git-subtree. I downloaded the git sources, and did an install from the contrib/subtree folder, with make;sudo make -prefix=/XXX install where XXX is the path containing bin/git. Might be worth noting that twist for others who attempt it.

The only part of the build that required a bit of research was the installation of git-subtree. I downloaded the git sources, and did an install from the contrib/subtree folder, with `make;sudo make -prefix=/XXX install` where XXX is the path containing `bin/git`. Might be worth noting that twist for others who attempt it.

emidoots commented

2021-11-06 17:44:18 +00:00

(Migrated from github.com)

@meshula can you clarify why you needed to install Git from source? AFAIK, only I myself should ever need to use git subtree, nobody else should ever need to.

I'd wait until I can get the .mm change landed, then Zig's CI will build artifacts for you.

@meshula can you clarify why you needed to install Git from source? AFAIK, only I myself should ever need to use `git subtree`, nobody else should ever need to. I'd wait until I can get the `.mm` change landed, then Zig's CI will build artifacts for you.

meshula commented

2021-11-06 19:27:33 +00:00

(Migrated from github.com)

Here's what I did.

git clone mach
zig build (didn't work, glfw not found, grep build system discover pull-subrepos.sh)
cd dev;./pull-subrepos.sh (complains about subtree because of github.com/hexops/mach@9f906cfd94/dev/pull-subrepos.sh (L6)
install git subtree as above, try pull-suprepos again (git loses it's mind, zomg)
delete mach; reclone
cd dev;./pull-subrepos.sh (succeeds)
cd ..;zig build succeeds in building engine and glfw
cd webgpu; zig build gets as far as noted above.

I was making it up as I went along since I wasn't sure what the intended build process actually is, and I haven't used git in a monorepo context in this way before. Maybe I missed an obvious (to one practiced in the monorepo arts ;) ) workflow step.

Here's what I did. 1. `git clone mach` 2. `zig build` (didn't work, glfw not found, `grep` build system discover pull-subrepos.sh) 3. `cd dev;./pull-subrepos.sh` (complains about subtree because of https://github.com/hexops/mach/blob/9f906cfd944a2dc9c5385a9e08231aa5c6f46521/dev/pull-subrepos.sh#L6 5. install git subtree as above, try `pull-suprepos` again (git loses it's mind, zomg) 6. delete mach; reclone 7. `cd dev;./pull-subrepos.sh` (succeeds) 8. `cd ..;zig build` succeeds in building engine and glfw 9. `cd webgpu; zig build` gets as far as noted above. I was making it up as I went along since I wasn't sure what the intended build process actually is, and I haven't used git in a monorepo context in this way before. Maybe I missed an obvious (to one practiced in the monorepo arts ;) ) workflow step.

emidoots commented

2021-11-06 19:30:40 +00:00

(Migrated from github.com)

zig build (didn't work, glfw not found,

I'd love more details on this part, what error did you get? GLFW is committed to the repository the same as any other file, so you really shouldn't have ever needed to touch git subtree at all. There isn't any magic here, git subtree just helps us with copying files back and forth between two repositories.

> zig build (didn't work, glfw not found, I'd love more details on this part, what error did you get? GLFW is committed to the repository the same as any other file, so you really shouldn't have ever needed to touch `git subtree` at all. There isn't any magic here, `git subtree` just helps us with copying files back and forth between two repositories.

meshula commented

2021-11-06 19:32:17 +00:00

(Migrated from github.com)

hmm, I think i have another mac without git subtree installed. standby.

meshula commented

2021-11-06 20:01:04 +00:00

(Migrated from github.com)

ok, all of my macs except that one have subtree installed, so (1) there is likely something wrong with git on that machine, probably I've hacked it too much with aliases and helper things. (2) all my other machines function as you describe; I can clone and zig build in the root directory, and in the dawn branch, in the webgpu directory, with the same .mm issue.

All right, sorry for the noise, but glad to have worked through this.

ok, all of my macs except that one have subtree installed, so (1) there is likely something wrong with git on that machine, probably I've hacked it too much with aliases and helper things. (2) all my other machines function as you describe; I can clone and zig build in the root directory, and in the dawn branch, in the webgpu directory, with the same .mm issue. All right, sorry for the noise, but glad to have worked through this.

emidoots commented

2021-11-06 21:52:57 +00:00

(Migrated from github.com)

Perfect! Glad to hear it.

Rows
Columns

gpu: build idiomatic Zig interface to WebGPU, abstract away native+browser support #21