hexops/mach

Fork 0

gpu-dawn: Building on Windows - FileNotFound error in gpu-dawn build #315

New issue

Closed

opened 2022-05-29 05:08:39 +00:00 by cocoademon · 9 comments

cocoademon commented

2022-05-29 05:08:39 +00:00

(Migrated from github.com)

This may be something very obvious that I have done wrong, being completely new to Zig.

I've tried running the homepage commands on Windows, and ran into the developer mode problem.
After working around this, and also updating all the .paths to .source in the build files, I was able to compile the build file.

However, the build fails at:

error.Unexpected: GetLastError(267): The directory name is invalid.

... [ snip zig stdlib stuff ] ...

C:\projects\mach\gpu\libs\mach-gpu-dawn\build.zig:425:45: 0x7ff6c52a7105 in @build.gpu.libs.mach-gpu-dawn.build.getCurrentGitCommit (build.obj)
    const result = try std.ChildProcess.exec(.{

(I can supply full trace if necessary)

This is a CreateProcess which tries to run Git - git is definitely 100% on the PATH, and I can run it from the same command line env that I'm running zig build from. (Although git is under Program Files - I haven't tried putting it in a dir with no spaces).

I know Zig historically had issues running windows programs that weren't in the CWD, but they were all closed over a year ago (eg https://github.com/ziglang/zig/issues/4120).

Is there anything obviously wrong I'm doing?

Running zig version 0.10.0-dev.2431+0e6285c8f

This may be something very obvious that I have done wrong, being completely new to Zig. I've tried running the homepage commands on Windows, and ran into the developer mode problem. After working around this, and also updating all the `.path`s to `.source` in the build files, I was able to compile the build file. However, the build fails at: ``` error.Unexpected: GetLastError(267): The directory name is invalid. ... [ snip zig stdlib stuff ] ... C:\projects\mach\gpu\libs\mach-gpu-dawn\build.zig:425:45: 0x7ff6c52a7105 in @build.gpu.libs.mach-gpu-dawn.build.getCurrentGitCommit (build.obj) const result = try std.ChildProcess.exec(.{ ``` (I can supply full trace if necessary) This is a CreateProcess which tries to run Git - `git` is definitely 100% on the PATH, and I can run it from the same command line env that I'm running `zig build` from. (Although `git` is under Program Files - I haven't tried putting it in a dir with no spaces). I know Zig historically had issues running windows programs that weren't in the CWD, but they were all closed over a year ago (eg https://github.com/ziglang/zig/issues/4120). Is there anything obviously wrong I'm doing? Running zig version 0.10.0-dev.2431+0e6285c8f

cocoademon commented

2022-05-29 05:17:30 +00:00

(Migrated from github.com)

Addendum - putting git in c:\git (and updating path) makes no difference.

Addendum - putting `git` in c:\git (and updating path) makes no difference.

d3m1gd commented

2022-05-29 05:21:20 +00:00

(Migrated from github.com)

i've seen similar error trying to build without git present. right now mach's build.zig doesn't produce a proper error if git is not seen by zig build.

d3m1gd commented

2022-05-29 05:30:08 +00:00

(Migrated from github.com)

there is a pr #316 to check for git.
can you please try to build again when it merges?

there is a pr #316 to check for git. can you please try to build again when it merges?

cocoademon commented

2022-05-29 05:35:36 +00:00

(Migrated from github.com)

Tried to build a minimum repro:

fn thisDir() []const u8 {
    return std.fs.path.dirname(@src().file) orelse ".";
}

fn getCurrentGitCommit(allocator: std.mem.Allocator) ![]const u8 {
    const result = try std.ChildProcess.exec(.{
        .allocator = allocator,
        .argv = &.{ "git", "version" },
        .cwd = thisDir(),
    });
    if (result.stdout.len > 0) return result.stdout[0 .. result.stdout.len - 1]; // trim newline
    return result.stdout;
}

pub fn main() !void {
    const allocator = std.heap.page_allocator;
    const stdout = std.io.getStdOut().writer();
    const commit = try getCurrentGitCommit(allocator);
    try stdout.print("Git commit: {s}\n", .{commit});
}

This worked fine. Git is present and runs ok.

Made me wonder if the error was in thisDir, and possibly related to the Windows symlink nonsense, so I changed into the gpu-dawn folder and ran zig build - it downloaded the dawn_x86...lib.gz and headers.json.gz and appeared to complete successfully.

I'll keep poking.

Tried to build a minimum repro: ```zig fn thisDir() []const u8 { return std.fs.path.dirname(@src().file) orelse "."; } fn getCurrentGitCommit(allocator: std.mem.Allocator) ![]const u8 { const result = try std.ChildProcess.exec(.{ .allocator = allocator, .argv = &.{ "git", "version" }, .cwd = thisDir(), }); if (result.stdout.len > 0) return result.stdout[0 .. result.stdout.len - 1]; // trim newline return result.stdout; } pub fn main() !void { const allocator = std.heap.page_allocator; const stdout = std.io.getStdOut().writer(); const commit = try getCurrentGitCommit(allocator); try stdout.print("Git commit: {s}\n", .{commit}); } ``` This worked fine. Git is present and runs ok. Made me wonder if the error was in `thisDir`, and possibly related to the Windows symlink nonsense, so I changed into the `gpu-dawn` folder and ran `zig build` - it downloaded the dawn_x86...lib.gz and headers.json.gz and appeared to complete successfully. I'll keep poking.

emidoots commented

2022-05-29 12:32:11 +00:00

(Migrated from github.com)

@cocoademon I'm pretty sure this is the issue you're running into, in the FAQ: https://github.com/hexops/mach/blob/main/doc/known-issues.md#windows-file-not-found

I am hoping to eventually eliminate all symlinks from the codebase to avoid this issue on Windows (their symlink approach is crazy bad out of the box.) Where could I have linked these docs so that you might have seen them?

@cocoademon I'm pretty sure this is the issue you're running into, in the FAQ: https://github.com/hexops/mach/blob/main/doc/known-issues.md#windows-file-not-found I am hoping to eventually eliminate all symlinks from the codebase to avoid this issue on Windows (their symlink approach is crazy bad out of the box.) Where could I have linked these docs so that you might have seen them?

cocoademon commented

2022-05-29 22:46:37 +00:00

(Migrated from github.com)

@slimsag I think you're right, but there are two problems here.

I followed the "developer mode" steps to as the first thing I did. That FAQ was the first thing I found, so no need to do a better job linking it. (I didn't follow the WSL approach as I didn't want to go through another installation hurdle)

I created a minimal build using zig init-exe and was able to reproduce the error where git wasn't found on the path but only during zig build.
This appears to have been a known problem: https://github.com/ziglang/zig/issues/10561 with a resolution that might not have landed in master yet.

Switching my Windows 10 Path env var to PATH instead solved it in the minimal repro, and now gives me a different error while building Mach:

C:\projects\mach>zig build run-example-boids
error.Unexpected: GetLastError(267): The directory name is invalid.

[snip]
                                       ^
D:\zig\lib\std\os\windows.zig:1598:49: 0x7ff6b27046af in td.os.windows.CreateProcessW (build.obj)
            else => |err| return unexpectedError(err),
                                                ^
[ snip ]
                       ^
C:\projects\mach\gpu\libs\mach-gpu-dawn\build.zig:425:45: 0x7ff6b27070e5 in @build.gpu.libs.mach-gpu-dawn.build.getCurrentGitCommit (build.obj)
    const result = try std.ChildProcess.exec(.{

"Directory name invalid" in CreateProcessW is weird, and it's caused by setting the CWD to an invalid path.
I threw a std.debug.print into the getCurrentGitCommit function, and found that @src().file is set to the full symlink dir: C:\projects\mach\gpu\libs\mach-gpu-dawn\build.zig.

I can't CD into this path, so I presume that there needs to be some canonicalization.

As a test fix, I hardcoded thisDir in gpu-dawn/build.zig to the following:

fn thisDir() []const u8 {
    return "C:\\projects\\mach\\gpu-dawn";
    // std.fs.path.dirname(@src().file) orelse ".";
}

and Success! It's running some Boids!

It seems like Windows can't have a CWD with a symlink in it when calling CreateProcessW. I'm trying to figure out a more robust solution - there's no obvious "canonicalize path" function in Windows.

Things I've tried instead of hardcoding the gpu-dawn path:

Using std.os.realpath. I couldn't get this working but I blame being a total Zig noob.
Sidestepping the dirname(@src().file)) and just using ".".

@slimsag I think you're right, but there are two problems here. I followed the "developer mode" steps to as the first thing I did. That FAQ was the first thing I found, so no need to do a better job linking it. (I didn't follow the WSL approach as I didn't want to go through another installation hurdle) I created a minimal build using `zig init-exe` and was able to reproduce the error where `git` wasn't found on the path but only during `zig build`. This appears to have been a known problem: https://github.com/ziglang/zig/issues/10561 with a resolution that might not have landed in master yet. Switching my Windows 10 Path env var to PATH instead solved it in the minimal repro, and now gives me a different error while building Mach: ``` C:\projects\mach>zig build run-example-boids error.Unexpected: GetLastError(267): The directory name is invalid. [snip] ^ D:\zig\lib\std\os\windows.zig:1598:49: 0x7ff6b27046af in td.os.windows.CreateProcessW (build.obj) else => |err| return unexpectedError(err), ^ [ snip ] ^ C:\projects\mach\gpu\libs\mach-gpu-dawn\build.zig:425:45: 0x7ff6b27070e5 in @build.gpu.libs.mach-gpu-dawn.build.getCurrentGitCommit (build.obj) const result = try std.ChildProcess.exec(.{ ``` "Directory name invalid" in CreateProcessW is weird, and it's caused by setting the CWD to an invalid path. I threw a std.debug.print into the `getCurrentGitCommit` function, and found that `@src().file` is set to the full symlink dir: `C:\projects\mach\gpu\libs\mach-gpu-dawn\build.zig`. I can't CD into this path, so I presume that there needs to be some canonicalization. As a test fix, I hardcoded `thisDir` in gpu-dawn/build.zig to the following: ``` fn thisDir() []const u8 { return "C:\\projects\\mach\\gpu-dawn"; // std.fs.path.dirname(@src().file) orelse "."; } ``` and **Success! It's running some Boids!** It seems like Windows can't have a CWD with a symlink in it when calling CreateProcessW. I'm trying to figure out a more robust solution - there's no obvious "canonicalize path" function in Windows. Things I've tried instead of hardcoding the gpu-dawn path: - Using std.os.realpath. I couldn't get this working but I blame being a total Zig noob. - Sidestepping the `dirname(@src().file))` and just using ".".

emidoots commented

2022-05-30 17:26:39 +00:00

(Migrated from github.com)

OK, makes sense, thanks for reporting this. I do think we could solve this with std.os.realpath - but for me this just reiterates, again, how bad/annoying symlinks are to deal with on Windows.

I'll just aim to eliminate all symlinks. If someone wants to PR the std.os.realpath solution feel free to!

OK, makes sense, thanks for reporting this. I do think we could solve this with `std.os.realpath` - but for me this just reiterates, again, how bad/annoying symlinks are to deal with on Windows. I'll just aim to eliminate all symlinks. If someone wants to PR the `std.os.realpath` solution feel free to!

alichraghi commented

2022-09-29 15:49:42 +00:00

(Migrated from github.com)

thisDir (sdkPath now) is mostly used in comptime concats, while std.os.realpath requires runtime computation

`thisDir` (`sdkPath` now) is mostly used in comptime concats, while `std.os.realpath` requires runtime computation

👍 1

emidoots commented

2022-10-18 19:33:23 +00:00

(Migrated from github.com)

Closing in favor of https://github.com/hexops/mach/issues/587