hexops/mach

Fork 0

gpu-dawn: standalone usage: headers are hard to find, need simple C webgpu.h example #605

New issue

Closed

opened 2022-10-30 18:31:36 +00:00 by ghost · 5 comments

ghost commented

2022-10-30 18:31:36 +00:00

(Migrated from github.com)

Hi slimsag,

You mention:

If you intend to actually call the WebGPU API, you should build these two source files as part of your application:

dawn_proc.c
webgpu_dawn_native_proc.cpp

But dawn_proc.c references 'dawn/dawn_proc.h' which is nowhere to be found.

If you include a single working hello world triangle C language example that compiles manually with, e.g., 'zig cc' or MSYS2 gcc/clang that would be extremely helpful to those of us who are not build engineers.

I tried building dawn from source following exactly your instructions with newest zig nightly to see if this process would generate the missing glue. I need to build for Windows from Linux so I ran:

/mnt/c/src/mach-gpu-dawn$ zig build -Dtarget=x86_64-windows-gnu

This eventually failed trying to compile DX dependencies, first few build errors were:

/mnt/c/src/mach-gpu-dawn/libs/DirectXShaderCompiler/lib/Support/TimeValue.cpp:1:1: error: unable to build C object: clang exited with code 1
/mnt/c/src/mach-gpu-dawn/libs/DirectXShaderCompiler/lib/Support/ThreadLocal.cpp:1:1: error: unable to build C object: clang exited with code 1
/mnt/c/src/mach-gpu-dawn/libs/DirectXShaderCompiler/lib/Support/Threading.cpp:1:1: error: unable to build C object: clang exited with code 1
[...]

I've spent several hours at this.

Alas, I'm forced to give up on Zig for now due to my lack of sufficient problem solving skills. On Windows MSYS2 I've built SDL2 with DX9, DX11, and DX12 SDL renderer backends and it all 'just works'.

I built Google Angle before going through the whole depot_tools rigamarole and it did, in the end, 'just work'.

From this day forward, if some Zig evangelist on HN says Zig builds 'just work', I'm sharing my counter-experience that for me it did not 'just work' even after hours upon hours of frustration.

But if I can compile a single C language webgpu.h hello triangle example for Windows then I will share that at least I got that much working. There is a repo that has a bunch of C webgpu.h Dawn examples but they only build on Linux for Linux, no Windows build:

https://github.com/samdauwe/webgpu-native-examples

Hi slimsag, You mention: > If you intend to actually call the WebGPU API, you should build these two source files as part of your application: > > dawn_proc.c > webgpu_dawn_native_proc.cpp But dawn_proc.c references 'dawn/dawn_proc.h' which is nowhere to be found. If you include a single working hello world triangle C language example that compiles manually with, e.g., 'zig cc' or MSYS2 gcc/clang that would be extremely helpful to those of us who are not build engineers. I tried building dawn from source following exactly your instructions with newest zig nightly to see if this process would generate the missing glue. I need to build for Windows from Linux so I ran: > /mnt/c/src/mach-gpu-dawn$ zig build -Dtarget=x86_64-windows-gnu This eventually failed trying to compile DX dependencies, first few build errors were: > /mnt/c/src/mach-gpu-dawn/libs/DirectXShaderCompiler/lib/Support/TimeValue.cpp:1:1: error: unable to build C object: clang exited with code 1 > /mnt/c/src/mach-gpu-dawn/libs/DirectXShaderCompiler/lib/Support/ThreadLocal.cpp:1:1: error: unable to build C object: clang exited with code 1 > /mnt/c/src/mach-gpu-dawn/libs/DirectXShaderCompiler/lib/Support/Threading.cpp:1:1: error: unable to build C object: clang exited with code 1 > [...] I've spent several hours at this. Alas, I'm forced to give up on Zig for now due to my lack of sufficient problem solving skills. On Windows MSYS2 I've built SDL2 with DX9, DX11, and DX12 SDL renderer backends and it all 'just works'. I built Google Angle before going through the whole depot_tools rigamarole and it did, in the end, 'just work'. From this day forward, if some Zig evangelist on HN says Zig builds 'just work', I'm sharing my counter-experience that for me it did not 'just work' even after hours upon hours of frustration. But if I can compile a single C language webgpu.h hello triangle example for Windows then I will share that at least I got that much working. There is a repo that has a bunch of C webgpu.h Dawn examples but they only build on Linux for Linux, no Windows build: [https://github.com/samdauwe/webgpu-native-examples](https://github.com/samdauwe/webgpu-native-examples)

emidoots commented

2022-10-31 16:30:26 +00:00

(Migrated from github.com)

Hi there

I think that you intended your message in a different way, so I just want to communicate how I interpreted it. I was drinking my morning coffee, saw your issue, and at first I got excited someone was going to try using Zig/mach-gpu-dawn like this. That's a first! I can help with this :) But then I read that last part about "From this day forward, ..." and thought, well, that's kind of F'd. It sounds to me a bit like "fix my issue or else I'll tell people Zig/Mach is shit!" and that was a little hurtful given I haven't even been given the chance to try and help yet.

I do get you're frustrated - modern build systems make me go bonkers too (I sometimes wonder how many software engineers get 'burnt out' just due to shitty build systems), I'm trying to solve this problem every day after hours for free, while working a difficult startup day job, so hopefully the fact that I'm personally passionate about solving this shitty experience for both you and me comes through here :)

The last thing I want to state (since this is a good time to do so) is that the Mach project, it's issue tracker, etc. is for code only. Anything that's not a direct discussion of code/tech, in general, is not welcome. It's just too time-consuming to manage, can baloon quickly into a nasty issue that appears on HN or similar, and after all, we're here for cool code/tech, right? I do absolutely encourage you to share your real experience with Zig and/or Mach other places, though, especially if it's bad or just "doesn't live up to the hype for me!" because I think we need more real experienced voices in our communities rather than endless hype chasing like we see in some other ecosystems cough frontend cough.

About the actual issues at hand..

..regarding Linux -> Windows cross compilation of Dawn from source, you are correct that is broken currently - I am sorry to say:

Cross-compiling to Windows from any other OS with a case-insensitive filesystem works (e.g. macOS), but since Linux is typically a case-sensitive FS it does not currently.
This is because Microsoft is inconsistent themselves about the case sensitivity of standard Windows header files. For example, the file may be named objidl.h on disk, while being #include'd under both ObjIdl.h, objidl.h, and OBJIDL.h throughout Microsoft code. It's some absolute insanity on their part, which I haven't yet managed to untangle.
I would like to fix this soon; it's being tracked here: https://github.com/hexops/mach/issues/489

..regarding the missing source files (your original question):

I would love to improve this to make it easier for use-cases like yours to consume. Right now mach-gpu-dawn has been mostly optimized torwards use in Mach engine itself; but I'd love to improve this because I do think it'd be a nice way to build projects like the one you linked, cross-compile them, etc.
That said, it's not too bad today. The files are there, just not easily discoverable (apologies for that.) The sources files you need are all going to be located here:
- https://github.com/hexops/dawn/tree/generated-2022-08-06/out/Debug/gen
- https://github.com/hexops/dawn/tree/generated-2022-08-06/include

The complete set of sources and their dependencies needed is:

Files you need to build for API access:
- dawn_proc.c (you must build for API access)
  - dawn/dawn_proc.h
    - dawn/dawn_proc_table.h
    - dawn/webgpu.h
- webgpu_dawn_native_proc.cpp (you must build for API access)
  - dawn/webgpu.h
The header you should include to use the WebGPU API (note that Dawn webgpu.h and other webgpu.h versions are not strictly ABI compatible.):
- dawn/webgpu.h

Final thought: It's genuinely really cool you're giving this a go! If you want reach out to me on Discord slimsag#2321 I am happy to help more with this if you get stuck. If someone like you is actually using mach-gpu-dawn for this, I am also happy to put a bit of time into e.g. packaging up those headers as a tarball or something in the CI pipeline so it's easier to fetch them if that's better for you. I would love to have a triangle C example using zig cc, I don't think it'd be too hard to add, I just haven't had time to write up one.

Hi there I think that you intended your message in a different way, so I just want to communicate how I interpreted it. I was drinking my morning coffee, saw your issue, and at first I got excited someone was going to try using Zig/mach-gpu-dawn like this. That's a first! I can help with this :) But then I read that last part about "From this day forward, ..." and thought, well, that's kind of F'd. It sounds to me a bit like "fix my issue or else I'll tell people Zig/Mach is shit!" and that was a little hurtful given I haven't even been given the chance to try and help yet. I do get you're frustrated - modern build systems make me go bonkers too (I sometimes wonder how many software engineers get 'burnt out' just due to shitty build systems), I'm trying to solve this problem every day after hours for free, while working a difficult startup day job, so hopefully the fact that I'm personally passionate about solving this shitty experience for both you and me comes through here :) The last thing I want to state (since this is a good time to do so) is that the Mach project, it's issue tracker, etc. is for code only. Anything that's not a direct discussion of code/tech, in general, is not welcome. It's just too time-consuming to manage, can baloon quickly into a nasty issue that appears on HN or similar, and after all, we're here for cool code/tech, right? I do absolutely encourage you to share your real experience with Zig and/or Mach other places, though, especially if it's bad or just "doesn't live up to the hype for me!" because I think we need more real experienced voices in our communities rather than endless hype chasing like we see in some other ecosystems *cough* frontend *cough*. --- About the actual issues at hand.. --- ..regarding Linux -> Windows cross compilation of Dawn from source, you are correct that is broken currently - I am sorry to say: * Cross-compiling to Windows from any other OS with a case-insensitive filesystem works (e.g. macOS), but since Linux is typically a case-sensitive FS it does not currently. * This is because Microsoft is inconsistent themselves about the case sensitivity of standard Windows header files. For example, the file may be named `objidl.h` on disk, while being `#include`'d under both `ObjIdl.h`, `objidl.h`, and `OBJIDL.h` throughout Microsoft code. It's some absolute insanity on their part, which I haven't yet managed to untangle. * I would like to fix this soon; it's being tracked here: https://github.com/hexops/mach/issues/489 --- ..regarding the missing source files (your original question): * I would love to improve this to make it easier for use-cases like yours to consume. Right now mach-gpu-dawn has been mostly optimized torwards use in Mach engine itself; but I'd love to improve this because I do think it'd be a nice way to build projects like the one you linked, cross-compile them, etc. * That said, it's not *too* bad today. The files _are there_, just not easily discoverable (apologies for that.) The sources files you need are all going to be located here: * https://github.com/hexops/dawn/tree/generated-2022-08-06/out/Debug/gen * https://github.com/hexops/dawn/tree/generated-2022-08-06/include The complete set of sources and their dependencies needed is: * Files you need to build for API access: * [dawn_proc.c](https://raw.githubusercontent.com/hexops/dawn/generated-2022-08-06/out/Debug/gen/src/dawn/dawn_proc.c) (you must build for API access) * [dawn/dawn_proc.h](https://raw.githubusercontent.com/hexops/dawn/generated-2022-08-06/include/dawn/dawn_proc.h) * [dawn/dawn_proc_table.h](https://raw.githubusercontent.com/hexops/dawn/generated-2022-08-06/out/Debug/gen/include/dawn/dawn_proc_table.h) * [dawn/webgpu.h](https://raw.githubusercontent.com/hexops/dawn/generated-2022-08-06/out/Debug/gen/include/dawn/webgpu.h) * [webgpu_dawn_native_proc.cpp](https://raw.githubusercontent.com/hexops/dawn/generated-2022-08-06/out/Debug/gen/src/dawn/native/webgpu_dawn_native_proc.cpp) (you must build for API access) * [dawn/webgpu.h](https://raw.githubusercontent.com/hexops/dawn/generated-2022-08-06/out/Debug/gen/include/dawn/webgpu.h) * The header you should include to use the WebGPU API (note that Dawn `webgpu.h` and other `webgpu.h` versions are not strictly ABI compatible.): * [dawn/webgpu.h](https://raw.githubusercontent.com/hexops/dawn/generated-2022-08-06/out/Debug/gen/include/dawn/webgpu.h) --- Final thought: It's genuinely really cool you're giving this a go! If you want reach out to me on Discord slimsag#2321 I am happy to help more with this if you get stuck. If someone like you is actually using mach-gpu-dawn for this, I am also happy to put a bit of time into e.g. packaging up those headers as a tarball or something in the CI pipeline so it's easier to fetch them if that's better for you. I would love to have a triangle C example using `zig cc`, I don't think it'd be too hard to add, I just haven't had time to write up one.

ghost commented

2022-11-01 07:47:29 +00:00

(Migrated from github.com)

Hi slimsag,

Thanks for your detailed reply. I did word my issue very poorly and I do apologize. Insofar as we live in the Age of Entitlement I ought have been more clear that I'm not owed the hard work of others for free.

You come to the heart of the matter when you say this:

I sometimes wonder how many software engineers get 'burnt out' just due to shitty build systems

This is me.

Solutions like NPM's package-lock.json and Rust's Cargo.lock aren't perfect but they prevent build breaks well enough that developers don't abandon ship for something else.

Debian dpkg was created in the 1990s and mostly 'just works' solving dependencies for both binary and source packages.

So, here we are in the 2020s, it's $CURRENT_YEAR, and Zig hasn't yet solved build breakage frustration as well as its predecessors.

None of this is your fault, of course.

I just found this open issue, "package manager", from Apr 22, 2018:

https://github.com/ziglang/zig/issues/943

Andrew Kelley started Zig in 2015. It's almost 2023. For comparison, how long did Rust take to release Cargo? Debian to come up with dpkg?

I decided to try Zig because of HN evangelism. So my remark about saying I will henceforth be honest about my lousy Zig experience on HN was not intended to be a sulky threat. My Rust experience hasn't been all that the Rust Evangelism Strike Force promised me, either. But getting Rust wgpu samples compiling and running did indeed 'just work' the first try thanks to Cargo 'just working'. So the next time a stupid Rust vs. Zig war breaks out (or tabs vs. spaces, or Emacs vs. vi) I'll be one of the Opinionated Idiots shouting over the voices of others with my woeful tale about how Zig failed to do the thing I wanted Zig to do and therefore Zig eternally forever sucks and no one should ever use it ever.

I'm just kidding, of course. If I have anything to say at all, I'll mention Zig folks are very friendly and helpful at going 'above and beyond', as you've demonstrated, so thank you.

Clearer up-front communication of current Zig instability would help better manage developer expectations. E.g., if ziglang.org made it crystal clear on the front page that Zig is unstable and lacking industrial strength dependency management rather than instead implying Zig is a robust and maintainable C/C++ replacement as if it had already achieved this, then that would spare many a developer from hours of frustration.

But this is something Andrew Kelley should do, not you.

Zig will get there eventually. One day we'll have Year of the Linux Desktop and one day we'll have Year of Zig Build Breaks Solved.

Again, sorry for coming across as sulky and rude. I appreciate your open source contributions and sacrifices, which are well above and beyond my own.

Hi slimsag, Thanks for your detailed reply. I did word my issue very poorly and I do apologize. Insofar as we live in the Age of Entitlement I ought have been more clear that I'm not owed the hard work of others for free. You come to the heart of the matter when you say this: > I sometimes wonder how many software engineers get 'burnt out' just due to shitty build systems This is me. Solutions like NPM's package-lock.json and Rust's Cargo.lock aren't perfect but they prevent build breaks well enough that developers don't abandon ship for something else. Debian dpkg was created in the 1990s and mostly 'just works' solving dependencies for both binary and source packages. So, here we are in the 2020s, it's $CURRENT_YEAR, and Zig hasn't yet solved build breakage frustration as well as its predecessors. None of this is your fault, of course. I just found this open issue, "package manager", from Apr 22, 2018: [https://github.com/ziglang/zig/issues/943](https://github.com/ziglang/zig/issues/943) Andrew Kelley started Zig in 2015. It's almost 2023. For comparison, how long did Rust take to release Cargo? Debian to come up with dpkg? I decided to try Zig because of HN evangelism. So my remark about saying I will henceforth be honest about my lousy Zig experience on HN was not intended to be a sulky threat. My Rust experience hasn't been all that the Rust Evangelism Strike Force promised me, either. But getting Rust wgpu samples compiling and running did indeed 'just work' the first try thanks to Cargo 'just working'. So the next time a stupid Rust vs. Zig war breaks out (or tabs vs. spaces, or Emacs vs. vi) I'll be one of the Opinionated Idiots shouting over the voices of others with my woeful tale about how Zig failed to do the thing I wanted Zig to do and therefore Zig eternally forever sucks and no one should ever use it ever. I'm just kidding, of course. If I have anything to say at all, I'll mention Zig folks are very friendly and helpful at going 'above and beyond', as you've demonstrated, so thank you. Clearer up-front communication of current Zig instability would help better manage developer expectations. E.g., if ziglang.org made it crystal clear on the front page that Zig is *unstable* and lacking industrial strength dependency management rather than instead implying Zig is a robust and maintainable C/C++ replacement as if it had already achieved this, then that would spare many a developer from hours of frustration. But this is something Andrew Kelley should do, not you. Zig will get there eventually. One day we'll have Year of the Linux Desktop and one day we'll have Year of Zig Build Breaks Solved. Again, sorry for coming across as sulky and rude. I appreciate your open source contributions and sacrifices, which are well above and beyond my own.

desttinghim commented

2022-11-01 19:26:29 +00:00

(Migrated from github.com)

Clearer up-front communication of current Zig instability would help better manage developer expectations. E.g., if ziglang.org made it crystal clear on the front page that Zig is unstable and lacking industrial strength dependency management rather than instead implying Zig is a robust and maintainable C/C++ replacement as if it had already achieved this, then that would spare many a developer from hours of frustration.

But this is something Andrew Kelley should do, not you.

An issue on the https://github.com/ziglang/www.ziglang.org repo on more clearly noting Zig's current instability on the homepage would probably be appreciated.

Andrew Kelley started Zig in 2015. It's almost 2023. For comparison, how long did Rust take to release Cargo? Debian to come up with dpkg?

This made me curious, so I searched it up. Graydon Hoare started work on Rust in 2006, and it became officially sponsered by Mozilla sometime in 2010. Cargo wasn't introduced until 2014, a little after the Rust 1.0 release. Using those dates, it took 8 years for Rust to get a package manager. Zig will likely end up in the same ballpark if the current roadmap pans out.

> Clearer up-front communication of current Zig instability would help better manage developer expectations. E.g., if ziglang.org made it crystal clear on the front page that Zig is _unstable_ and lacking industrial strength dependency management rather than instead implying Zig is a robust and maintainable C/C++ replacement as if it had already achieved this, then that would spare many a developer from hours of frustration. > > But this is something Andrew Kelley should do, not you. An issue on the https://github.com/ziglang/www.ziglang.org repo on more clearly noting Zig's current instability on the homepage would probably be appreciated. > Andrew Kelley started Zig in 2015. It's almost 2023. For comparison, how long did Rust take to release Cargo? Debian to come up with dpkg? This made me curious, so I searched it up. Graydon Hoare started work on Rust in 2006, and it became officially sponsered by Mozilla sometime in 2010. Cargo wasn't introduced until 2014, a little after the Rust 1.0 release. Using those dates, it took 8 years for Rust to get a package manager. Zig will likely end up in the same ballpark if the current roadmap pans out.

citruslee commented

2022-12-07 09:44:38 +00:00

(Migrated from github.com)

hey @slimsag I would love to pick up going through providing an example of a triangle in raw webgpu in zig, but I have to agree with @dev-muppet s point that the directions are a bit unclear how to get the raw webgpu calls in mach-gpu built. Could I ask you kindly to provide some steps to go through, and I will try my best to pick it up from there? I would be very invested in this, since me and my buddies are working on a demo engine/tool for our upcoming demoscene productions (go check pouet.net if you are interested in what we are doing) and we would like to use webgpu and zig and hopefully through your library, since you already did the legwork for it for which we are already grateful <3

hey @slimsag I would love to pick up going through providing an example of a triangle in raw webgpu in zig, but I have to agree with @dev-muppet s point that the directions are a bit unclear how to get the raw webgpu calls in mach-gpu built. Could I ask you kindly to provide some steps to go through, and I will try my best to pick it up from there? I would be very invested in this, since me and my buddies are working on a demo engine/tool for our upcoming demoscene productions (go check [pouet.net ](https://www.pouet.net/groups.php?which=12600) if you are interested in what we are doing) and we would like to use webgpu and zig and hopefully through your library, since you already did the legwork for it for which we are already grateful <3

emidoots commented

2024-07-08 13:05:45 +00:00

(Migrated from github.com)

https://github.com/hexops/mach/issues/1166#issuecomment-2212990680