mach: decide on high-level abstraction API #309

New issue

Closed

opened 2022-05-26 17:59:09 +00:00 by emidoots · 33 comments

emidoots commented 2022-05-26 17:59:09 +00:00 (Migrated from github.com)

Copy link

This is speaking of the currently in-development mach API - what most people will use to build Mach applications.

Goals

• Provide an abstraction over GLFW, Browser APIs, and Android/iOS APIs, for input/windowing/etc.
• Provide a way to expose platform-specific behavior when neccesary.
• Should be usable in very modular ways:
  • "I just want a cross-platform window/input basically, I'll use WebGPU directly for rendering and pretty much nothing else Mach provides"
  • "I want features XY that Mach provides, but not Z"

State of things today

As it stands today, your main program is defined as an App struct. Mach consumes this, defines the main entrypoint and calls your application init function and so on. This is the same design from xq's excellent zero-graphics library.

We then have:

• App -> your application
• mach.Engine -> given to most functions and provides access to core engine functionality (the abstraction)
• mach.Timer -> cross platform timer that works in wasm
• engine.gpu_driver.swap_chain -> the swap chain
• engine.gpu_driver.device -> the actual gpu.Interface WebGPU API
• engine.core.pollEvent() -> event polling
• engine.core.setSizeLimits() -> window abstraction functions

The callbacks your App main.zig can define are:

• pub fn init(app: *App, engine: *mach.Engine) !void
• pub fn deinit(app: *App, engine: *mach.Engine) void
• pub fn update(app: *App, engine: *mach.Engine) !bool
• pub fn resize(app: *App, engine: *mach.Engine, width: u32, height: u32) !void

Open questions / what this issue is about

How should the API look?

This is speaking of the currently in-development `mach` API - what most people will use to build Mach applications. ## Goals * Provide an abstraction over GLFW, Browser APIs, and Android/iOS APIs, for input/windowing/etc. * Provide a way to expose platform-specific behavior when neccesary. * Should be usable in very modular ways: * "I just want a cross-platform window/input basically, I'll use WebGPU directly for rendering and pretty much nothing else Mach provides" * "I want features XY that Mach provides, but not Z" ## State of things today As it stands today, your main program is defined as an `App` struct. Mach consumes this, defines the main entrypoint and calls your application `init` function and so on. This is the same design from xq's excellent zero-graphics library. We then have: * `App` -> your application * `mach.Engine` -> given to most functions and provides access to core engine functionality (the abstraction) * `mach.Timer` -> cross platform timer that works in wasm * `engine.gpu_driver.swap_chain` -> the swap chain * `engine.gpu_driver.device` -> the actual `gpu.Interface` WebGPU API * `engine.core.pollEvent()` -> event polling * `engine.core.setSizeLimits()` -> window abstraction functions The callbacks your `App` `main.zig` can define are: * `pub fn init(app: *App, engine: *mach.Engine) !void` * `pub fn deinit(app: *App, engine: *mach.Engine) void` * `pub fn update(app: *App, engine: *mach.Engine) !bool` * `pub fn resize(app: *App, engine: *mach.Engine, width: u32, height: u32) !void` ## Open questions / what this issue is about How should the API look?

emidoots commented 2022-05-26 18:11:37 +00:00 (Migrated from github.com)

Copy link

I still need to find time to sit down and look into this more deeply, and see if my ideas here are actually practical / applicible or not. On the surface, here are my basic thoughts:

App concerns

I think pub const options in App is too magical. It can be very confusing to define const options and not know why the options aren't taking effect (forgot pub.)

I think for newcomers, especially those new to Zig, App will be very confusing in general. Main would be a better name, but I'm hoping we can find a better way to make the API look more like "pass an App struct to mach" (even if that's not what is actually happening) than "I define an App, it consumes it magically" - I need to look into this.

API nesting concerns

Things like engine.gpu_driver.device and engine.core.pollEvent are too verbose for my taste. I'm hoping we can reduce these to something more like:

Before	After
engine.gpu_driver.swap_chain	engine.swap_chain
engine.gpu_driver.device	engine.gpu
engine.core.pollEvent	engine.pollEvent
engine.core.setSizeLimits	engine.setSizeLimits

(not a concrete proposal, this might introduce other issues I'm not aware of.)

Callback concerns

One thing that I think will add to the confusion between App and Engine ("which one's mine?", "what's the difference?") is that callbacks get both app: *App, engine: *mach.Engine right now. I think if we can reduce this to say just app: *App and have engine: *mach.Engine be a field of app that could be better perhaps.

Disclaimer

I really haven't thought out much of this, these are just surface-level thoughts. I want to sit down and review the code more carefully to get a better grasp of how these changes would impact things.

I still need to find time to sit down and look into this more deeply, and see if my ideas here are actually practical / applicible or not. On the surface, here are my basic thoughts: ## App concerns I think `pub const options` in `App` is too magical. It can be very confusing to define `const options` and not know why the options aren't taking effect (forgot `pub`.) I think for newcomers, especially those new to Zig, `App` will be very confusing in general. `Main` would be a better name, but I'm hoping we can find a better way to make the API look more like "pass an App struct to mach" (even if that's not what is _actually_ happening) than "I define an App, it consumes it magically" - I need to look into this. ## API nesting concerns Things like `engine.gpu_driver.device` and `engine.core.pollEvent` are too verbose for my taste. I'm hoping we can reduce these to something more like: | Before | After | |--------|-------| | `engine.gpu_driver.swap_chain` | `engine.swap_chain` | | `engine.gpu_driver.device` | `engine.gpu` | | `engine.core.pollEvent` | `engine.pollEvent` | | `engine.core.setSizeLimits` | `engine.setSizeLimits` | (not a concrete proposal, this might introduce other issues I'm not aware of.) ## Callback concerns One thing that I think will add to the confusion between `App` and `Engine` ("which one's mine?", "what's the difference?") is that callbacks get both `app: *App, engine: *mach.Engine` right now. I think if we can reduce this to say just `app: *App` and have `engine: *mach.Engine` be a field of `app` that could be better perhaps. ## Disclaimer I really haven't thought out much of this, these are just surface-level thoughts. I want to sit down and review the code more carefully to get a better grasp of how these changes would impact things.

iddev5 commented 2022-05-26 18:11:45 +00:00 (Migrated from github.com)

Copy link

Some things which we need to discuss: What exactly Engine is? Should it just be an interface to access core engine functionality (window, input, events, gpu init etc)? Or should it be a more complex collection of high level engine modules (like Physics, Renderer, Ecs) the collection of which may be influenced by comptime options.

With the first route, we can merge Core and GpuDriver and make Engine the replacement instead. So basically different backends will define their own Engine with their platform specific code (ofcouse this makes the name Engine confusing)

With the second option we get the advantage that engine features are automatically initialized and deinitialized and is part of one struct. If we DO NOT have it, an application would look like this:

world: ecs.World,
renderer: mach.Renderer,

pub fn init(app: *App, engine: *mach.Engine) {
    app.renderer = mach.Renderer.init();
    ...

But with this route, such things will be handled automatically (it will be part of Engine, like engine.ecs, engine.renderer and modules can be included and excluded based on comptime options.

---

One small input to the API: I think setSizeLimits should not be a function but rather a field in mach.Options. It should also be noted that this function does not and cannot do anything for wasm.

Some things which we need to discuss: What exactly Engine is? Should it just be an interface to access core engine functionality (window, input, events, gpu init etc)? Or should it be a more complex collection of high level engine modules (like Physics, Renderer, Ecs) the collection of which may be influenced by comptime options. With the first route, we can merge Core and GpuDriver and make Engine the replacement instead. So basically different backends will define their own ``Engine`` with their platform specific code (ofcouse this makes the name Engine confusing) With the second option we get the advantage that engine features are automatically initialized and deinitialized and is part of one struct. If we DO NOT have it, an application would look like this: ``` world: ecs.World, renderer: mach.Renderer, pub fn init(app: *App, engine: *mach.Engine) { app.renderer = mach.Renderer.init(); ... ``` But with this route, such things will be handled automatically (it will be part of Engine, like ``engine.ecs``, ``engine.renderer`` and modules can be included and excluded based on comptime options. --- One small input to the API: I think setSizeLimits should not be a function but rather a field in mach.Options. It should also be noted that this function does not and cannot do anything for wasm.

👍 2

iddev5 commented 2022-05-26 18:19:46 +00:00 (Migrated from github.com)

Copy link

pub const options ... is too magical

We can solve this in different ways. See https://github.com/hexops/mach/blob/main/src/Engine.zig#L84
What about making App.init return a mach.Options?

---

API nesting concerns

What you suggested is basically my first option.

---

Callback concerns

This needs to be discussed. Currently Engine is a magical struct and it would remain so. If let say some application dont define Engine as a field in their App, the whole program will basically break in unexpected ways. We can say that Engine is the core of mach right now.

> pub const options ... is too magical We can solve this in different ways. See https://github.com/hexops/mach/blob/main/src/Engine.zig#L84 What about making App.init return a mach.Options? --- > API nesting concerns What you suggested is basically my first option. --- > Callback concerns This needs to be discussed. Currently Engine is a magical struct and it would remain so. If let say some application dont define Engine as a field in their App, the whole program will basically break in unexpected ways. We can say that Engine is the core of mach right now.

👍 1

emidoots commented 2022-05-26 18:23:33 +00:00 (Migrated from github.com)

Copy link

Some things which we need to discuss: What exactly Engine is? Should it just be an interface to access core engine functionality (window, input, events, gpu init etc)? Or should it be a more complex collection of high level engine modules (like Physics, Renderer, Ecs) the collection of which may be influenced by comptime options.

This is a really great question, I can't believe I forgot to touch on this point - sorry!

I think it should just be "an interface to access core engine functionality (window, input, events, gpu init etc)" - with one caveat: it's also the interface to the ECS, which is how you compose everything and pick-and-choose which components of Mach you wish to use.

I have not implemented systems in our ECS yet in any meaningful way, and so I do not have a concrete API to demonstrate this, but I envision something similar to the Bevy engine API which looks largely like this:

    App::new()
        .add_startup_system(add_monsters)
        .add_system(renderer)
        .add_system(physics)
        .run();

Of course, ours would not be App::new() nor .run(), but the same basic idea behind "I have a place to add systems, and get systems"

Ours might look more like:

pub fn init(engine: *mach.Engine) !void {
    try engine.register("renderer", mach.renderer);
}

Where mach.renderer is an ECS system which discovers entities with Renderer components and renders them. All interaction with the renderer is done primarily through the ECS/entity interfaces, then, and only in rare cases do you need to access actual renderer functions.

This is my general thinking, not concrete ideas of course. There may be major issues I haven't thought of here.

> Some things which we need to discuss: What exactly Engine is? Should it just be an interface to access core engine functionality (window, input, events, gpu init etc)? Or should it be a more complex collection of high level engine modules (like Physics, Renderer, Ecs) the collection of which may be influenced by comptime options. This is a *really great* question, I can't believe I forgot to touch on this point - sorry! I think it should just be "an interface to access core engine functionality (window, input, events, gpu init etc)" - with one caveat: it's also the interface to the ECS, which is how you compose everything and pick-and-choose which components of Mach you wish to use. I have not implemented systems in our ECS yet in any meaningful way, and so I do not have a concrete API to demonstrate this, but I envision something similar to the Bevy engine API which looks largely like this: ``` App::new() .add_startup_system(add_monsters) .add_system(renderer) .add_system(physics) .run(); ``` Of course, ours would not be `App::new()` nor `.run()`, but the same basic idea behind "I have a place to add systems, and get systems" Ours might look more like: ```zig pub fn init(engine: *mach.Engine) !void { try engine.register("renderer", mach.renderer); } ``` Where `mach.renderer` is an ECS system which discovers entities with `Renderer` components and renders them. All interaction with the renderer is done primarily through the ECS/entity interfaces, then, and only in rare cases do you need to access actual renderer _functions_. This is my general thinking, not concrete ideas of course. There may be major issues I haven't thought of here.

emidoots commented 2022-05-26 18:24:33 +00:00 (Migrated from github.com)

Copy link

One small input to the API: I think setSizeLimits should not be a function but rather a field in mach.Options. It should also be noted that this function does not and cannot do anything for wasm.

Makes sense!

> One small input to the API: I think setSizeLimits should not be a function but rather a field in mach.Options. It should also be noted that this function does not and cannot do anything for wasm. Makes sense!

iddev5 commented 2022-05-26 18:35:11 +00:00 (Migrated from github.com)

Copy link

This is a really great question, I can't believe I forgot to touch on this point - sorry!

This makes sense. Though I suggest that modules should work as much as possible on comptime. We should discuss more about this on ECS channel though.

in rare cases do you need to access actual renderer functions

I dont think we can assume that. People might just want to use a conventional renderer for their project. I think what could be done is to have a base Renderer type. The ECS Renderer module uses this Renderer type internally. If someone wants, they can get a pointer to it and play around with it as they like*

Otherwise, if someone doesn't want to use the ECS Renderer at all, then they can basically initialize a Renderer themselves (as a field of App) as use it however they like.

Of course this whole part doesn't makes sense if we dont want to consider this usecase, but since mach is meant to be both a game engine and an application development framework, I think this should be considered.

> This is a really great question, I can't believe I forgot to touch on this point - sorry! This makes sense. Though I suggest that modules should work as much as possible on comptime. We should discuss more about this on ECS channel though. > in rare cases do you need to access actual renderer functions I dont think we can assume that. People might just want to use a conventional renderer for their project. I think what could be done is to have a base ``Renderer`` type. The ECS Renderer module uses this Renderer type internally. If someone wants, they can get a pointer to it and play around with it as they like* Otherwise, if someone doesn't want to use the ECS Renderer at all, then they can basically initialize a Renderer themselves (as a field of App) as use it however they like. Of course this whole part doesn't makes sense if we dont want to consider this usecase, but since mach is meant to be both a game engine and an application development framework, I think this should be considered.

emidoots commented 2022-05-26 18:49:36 +00:00 (Migrated from github.com)

Copy link

What about making App.init return a mach.Options?

Sounds like a good idea.

With the first route, we can merge Core and GpuDriver and make Engine the replacement instead. So basically different backends will define their own Engine with their platform specific code (ofcouse this makes the name Engine confusing)

Overall this approach sounds right to me. I am unsure if having each backend define their own Engine makes sense, but I really mean "unsure" here (it may well be the right way to do it.)

I don't think the name Engine would be confusing here, after all, the engine has to start somewhere. If not here, then what else would be called "engine"?

I suggest that modules should work as much as possible on comptime. We should discuss more about this on ECS channel though.

I would like this too. Let's chat more about this in the ECS channel for sure.

People might just want to use a conventional renderer for their project. I think what could be done is to have a base Renderer type. The ECS Renderer module uses this Renderer type internally. If someone wants, they can get a pointer to it and play around with it as they like*

This sounds generally good to me, but I do want to highlight that I would not be concerned if the only interface to Mach's built in Renderer is in fact via ECS only.

I think that our ECS approach should absolutely be good for an application development framework, not just for games. It may be confusing to hear this since ECS is mostly discussed in the context of games - but I think this can and will be a very valuable thing to have in place when developing regular UI applications.

I also agree that if someone doesn't want to use the builtin Mach renderer, they can of course have their own Renderer themselves as a field of App and use it however they like. Whether or not the builtin Renderer of Mach is suitable for this case, I don't know - but I don't think it would be a primary concern, more a 'nice to have'. This scenario would be more 'I am building my own renderer in WebGPU' I think.

> What about making App.init return a mach.Options? Sounds like a good idea. > With the first route, we can merge `Core` and `GpuDriver` and make `Engine` the replacement instead. So basically different backends will define their own `Engine` with their platform specific code (ofcouse this makes the name `Engine` confusing) Overall this approach sounds right to me. I am unsure if having each backend define their own `Engine` makes sense, but I really mean "unsure" here (it may well be the right way to do it.) I don't think the name Engine would be confusing here, after all, the engine has to start somewhere. If not here, then what else would be called "engine"? > I suggest that modules should work as much as possible on comptime. We should discuss more about this on ECS channel though. I would like this too. Let's chat more about this in the ECS channel for sure. > People might just want to use a conventional renderer for their project. I think what could be done is to have a base Renderer type. The ECS Renderer module uses this Renderer type internally. If someone wants, they can get a pointer to it and play around with it as they like* This sounds generally good to me, but I do want to highlight that I would not be concerned if the *only* interface to Mach's built in Renderer is in fact via ECS only. I think that our ECS approach should absolutely be good for an application development framework, not just for games. It may be confusing to hear this since ECS is mostly discussed in the context of games - but I think this can and will be a very valuable thing to have in place when developing regular UI applications. I also agree that if someone doesn't want to use the builtin Mach renderer, they can of course have their own Renderer themselves as a field of App and use it however they like. Whether or not the builtin Renderer of Mach is suitable for this case, I don't know - but I don't think it would be a primary concern, more a 'nice to have'. This scenario would be more 'I am building my own renderer in WebGPU' I think.

👍 1

d3m1gd commented 2022-05-27 01:09:27 +00:00 (Migrated from github.com)

Copy link

naming

The name Engine is a well known super high level term. Think of Unreal Engine. Something that gives you full set of tools to quickly build a game.

My proposal is to have a core type renamed to Core (unsurprisingly) and make it hold all the low level apis.

Core provides input handling, and exposes all the platform independent low level apis. not optional.
Engine provides ECS, physics, etc, is optional.

Core also holds platform specific fields that are never exposed to end user publicly (but might be still given access to if needed).

App api

With above naming in mind:

pub fn init(core: *Core) T; // required for all mach apps
pub fn update(app: *T, core: *Core) void; // optional, see below
pub fn deinit(app: *T, core: *Core) void; // optional
pub fn resize(app: *T, core: *Core) void; // optional

the T is fully user defined, and we get it in mach by examining init - the only required function, basically an entry point for user code.

Since T is user defined, options can be easily embedded by the user in init, if one wants to. Can be checked by examining T for Options.

Notice none of the functions return error union, and update does not return bool. Returning bool on update is not exactly descriptive. What about simply having a method Core.exit() and call if needed, which would call deinit if present. No confusion here. As for not returning errors - i envision no way on how a high level abstraction that App is supposed to be can fail on init or update. If they happen, there is no need to tell mach about it though, and user can simply crash manually or exit cleanly with an error through Core.exit() or maybe Core.exitWithError(), since if they are not recoverable by the App designer, mach can not do absolutely anything about those anyway.

The optional nature of update allows user wanting to go low level gpu-only approach to define update and do everything manually from there, while high level users will avoid it and get full mach, something like:

// comptime check, eliminates unused branch
if (@hasDecl("update") and properSignature()) {
    app.update(core);
} else {
    mach.physics();
    mach.ecs();
    mach.update();
}

## naming The name `Engine` is a well known super high level term. Think of Unreal Engine. Something that gives you full set of tools to quickly build a game. My proposal is to have a core type renamed to `Core` (unsurprisingly) and make it hold all the low level apis. `Core` provides input handling, and exposes all the platform independent low level apis. not optional. `Engine` provides ECS, physics, etc, is optional. `Core` also holds platform specific fields that are never exposed to end user publicly (but might be still given access to if needed). ## App api With above naming in mind: ``` pub fn init(core: *Core) T; // required for all mach apps pub fn update(app: *T, core: *Core) void; // optional, see below pub fn deinit(app: *T, core: *Core) void; // optional pub fn resize(app: *T, core: *Core) void; // optional ``` the `T` is fully user defined, and we get it in `mach` by examining `init` - the only required function, basically an entry point for user code. Since `T` is user defined, options can be easily embedded by the user in init, if one wants to. Can be checked by examining `T` for `Options`. Notice none of the functions return error union, and update does not return bool. Returning bool on update is not exactly descriptive. What about simply having a method `Core.exit()` and call if needed, which would call `deinit` if present. No confusion here. As for not returning errors - i envision no way on how a high level abstraction that `App` is supposed to be can fail on init or update. If they happen, there is no need to tell `mach` about it though, and user can simply crash manually or exit cleanly with an error through `Core.exit()` or maybe `Core.exitWithError()`, since if they are not recoverable by the `App` designer, `mach` can not do absolutely anything about those anyway. The optional nature of `update` allows user wanting to go low level gpu-only approach to define `update` and do everything manually from there, while high level users will avoid it and get full `mach`, something like: ``` // comptime check, eliminates unused branch if (@hasDecl("update") and properSignature()) { app.update(core); } else { mach.physics(); mach.ecs(); mach.update(); } ```

d3m1gd commented 2022-05-27 01:15:42 +00:00 (Migrated from github.com)

Copy link

actually can even give the user a way to indicate high or low level of mach by asking to provide either init(core: *mach.Core) or init(engine: *mach.Engine) but not both

actually can even give the user a way to indicate high or low level of `mach` by asking to provide either `init(core: *mach.Core)` *or* `init(engine: *mach.Engine)` but not both

iddev5 commented 2022-05-27 01:46:58 +00:00 (Migrated from github.com)

Copy link

In my last proposal of returning Options from Mach, I missed an important point that we are using Engine in init itself, and in future we would need to too. So back to drawing board.

In my last proposal of returning Options from Mach, I missed an important point that we are using Engine in init itself, and in future we would need to too. So back to drawing board.

👍 1

d3m1gd commented 2022-05-27 05:36:12 +00:00 (Migrated from github.com)

Copy link

api must follow principle of least astonishment.

now

const format = engine.gpu_driver.swap_chain_format;
...
engine.gpu_driver.swap_chain.?.present();

proposal

const format = engine.swap_chain.?.format; // notice format is a field now
...
engine.swap_chain.?.present();

those things are not directly related in code, but they are about the same subject, so must be more structurally unified. and additionally nesting is way distracting, and discoverability of methods suffer too.

Also, as to that non-null dereference. Any objection if i try refactoring to get rid of it internally?

api must follow principle of least astonishment. now ``` const format = engine.gpu_driver.swap_chain_format; ... engine.gpu_driver.swap_chain.?.present(); ``` proposal ``` const format = engine.swap_chain.?.format; // notice format is a field now ... engine.swap_chain.?.present(); ``` those things are not *directly* related in code, but they are about the same subject, so must be more structurally unified. and additionally nesting is way distracting, and discoverability of methods suffer too. Also, as to that non-null dereference. Any objection if i try refactoring to get rid of it internally?

iddev5 commented 2022-05-27 18:20:50 +00:00 (Migrated from github.com)

Copy link

We can safely remove bool from update(), I think I added it for recursion protection in wasm, but it doesnt happens in case of requestAnimationFrame,

---

Making update() optional is possible as with ECS, we wouldn't need this anyways, But having deinit() as optional is a bit tricky. First of all. in case of a single callback function (having just init) it means that it will have all the code including its internal deinits inside defer blocks. That means we cannot expect the program to run properly after init has returned. So we would need a new function called run() maybe inside Engine which will run the main update loop. But if update() is present, this function will be called implictly by the core engine systems.

This basically means that we can go for two routes: 1) have only init() OR 2) have atleast init(), deinit()

(1) gives the feeling of a more traditional application entry point, it is basically us recreation main() on top of main() (yes sounds a bit weird). We can also get rid of *App param in init() with (1)
(2) is similar to how we do things today and is less complex but *App param is necessary.

In any case we will have to have (2). So the question remains is do we want (1)?

We can safely remove bool from update(), I think I added it for recursion protection in wasm, but it doesnt happens in case of requestAnimationFrame, --- Making update() optional is possible as with ECS, we wouldn't need this anyways, But having deinit() as optional is a bit tricky. First of all. in case of a single callback function (having just init) it means that it will have all the code including its internal deinits inside defer blocks. That means we cannot expect the program to run properly after init has returned. So we would need a new function called run() maybe inside Engine which will run the main update loop. But if update() is present, this function will be called implictly by the core engine systems. This basically means that we can go for two routes: 1) have only init() OR 2) have atleast init(), deinit() (1) gives the feeling of a more traditional application entry point, it is basically us recreation main() on top of main() (yes sounds a bit weird). We can also get rid of *App param in init() with (1) (2) is similar to how we do things today and is less complex but *App param is necessary. In any case we will have to have (2). So the question remains is do we want (1)?

iddev5 commented 2022-05-27 18:26:30 +00:00 (Migrated from github.com)

Copy link

I am unsure if having each backend define their own Engine makes sense

Sorry forgot to reply and I worded the initial statement poorly. Currently with Core and GpuDriver, we have an internal field which is defined by different backends. I suggested that we combine Core and GpuDriver and move all of its functionality to Engine directly. So now we will have that internal: Platform.Type as a field inside of Engine instead of having it inside of another struct probably called Platform (reducing nesting).

The different platforms will define their own Platform.Type.

> I am unsure if having each backend define their own Engine makes sense Sorry forgot to reply and I worded the initial statement poorly. Currently with Core and GpuDriver, we have an internal field which is defined by different backends. I suggested that we combine Core and GpuDriver and move all of its functionality to Engine directly. So now we will have that ``internal: Platform.Type`` as a field inside of Engine instead of having it inside of another struct probably called Platform (reducing nesting). The different platforms will define their own ``Platform.Type``.

👍 1

emidoots commented 2022-05-28 04:34:31 +00:00 (Migrated from github.com)

Copy link

@iddev5

So we would need a new function called run() maybe inside Engine which will run the main update loop. But if update() is present, this function will be called implictly by the core engine systems.

This sounds generally good to me, but can you elaborate on the if update() is present, ... part? I'm not sure I fully understand it. I guess you're saying that if run() is used AND update is present then.. there's an edge case, and something happens then?

(1) gives the feeling of a more traditional application entry point, it is basically us recreation main() on top of main() (yes sounds a bit weird). We can also get rid of *App param in init() with (1)

I think this is worth doing. It does add more complexity, but giving the feeling of a more traditional entrypoint is a huge win for newcomers and first impressions in general. This sort of thing has untold benefits long term. That's not to say we have to do it today, but I do think it's the right tradeoff to choose here.

@iddev5 > So we would need a new function called run() maybe inside Engine which will run the main update loop. But if update() is present, this function will be called implictly by the core engine systems. This sounds generally good to me, but can you elaborate on the `if update() is present, ...` part? I'm not sure I fully understand it. I guess you're saying that if `run()` is used AND `update` is present then.. there's an edge case, and something happens then? > (1) gives the feeling of a more traditional application entry point, it is basically us recreation main() on top of main() (yes sounds a bit weird). We can also get rid of *App param in init() with (1) I think this is worth doing. It does add more complexity, but giving the feeling of a more traditional entrypoint is a huge win for newcomers and first impressions in general. This sort of thing has untold benefits long term. That's not to say we have to do it today, but I do think it's the right tradeoff to choose here.

iddev5 commented 2022-05-28 05:50:22 +00:00 (Migrated from github.com)

Copy link

there's an edge case

yes, basically we cannot have both run() being used and update being declared in the same program. Because run would be running a loop and will block from update() being ever called. I don't know how to actually solve this problem though in terms of compile time errors. If we don't provide something there's chance that users would accidentally call run() in their init() while they have an update() defined.

---

EDIT: SOLUTION: We can define run() as following:

pub fn run(engine: *Engine) !void {
    if (@hasDecl("update") and properSignature()) {
        @compileError("run() and update() both in same program is not allowed");
    } else {
        while (true) {
            engine.runOnce(); 
        }
    }
}

Also, the actual internal main would be:

pub fn main() !void {
    // Initialize engine and app
    // ...
    if (@hasDecl("update")) {
        while (true) {
            engine.runOnce();
            update();
        }
    }
}

> there's an edge case yes, basically we cannot have both ``run()`` being used and ``update`` being declared in the same program. Because run would be running a loop and will block from update() being ever called. I don't know how to actually solve this problem though in terms of compile time errors. If we don't provide something there's chance that users would accidentally call run() in their init() while they have an update() defined. --- EDIT: SOLUTION: We can define run() as following: ``` pub fn run(engine: *Engine) !void { if (@hasDecl("update") and properSignature()) { @compileError("run() and update() both in same program is not allowed"); } else { while (true) { engine.runOnce(); } } } ``` Also, the actual internal main would be: ``` pub fn main() !void { // Initialize engine and app // ... if (@hasDecl("update")) { while (true) { engine.runOnce(); update(); } } } ```

d3m1gd commented 2022-05-28 09:28:06 +00:00 (Migrated from github.com)

Copy link

if we have only one entry point, "main on top of main" and give engine to that and leave user decide what api to use.

manual low level way

pub fn my_main(engine: *mach.Engine) {
    engine.set_size(1024, 768)
    engine.set_vsync(.triple) 
    engine.recreate() // just an idea to avoid magic Options
    const back_buffer = engine.startFrame(); // can hide some lowest level things like that
    while (engine.pollEvents()) { }
    doManualRender();
    engine.swap()
}

full mach

pub fn my_main(engine: *mach.Engine) {
    engine.set_size(1024, 768)
    engine.set_vsync(.triple) 
    engine.recreate() // just an idea to avoid magic Options
    engine.setECSParams();
    engine.setPhysicsParams();
    engine.maybeSetSomeCallbacks();
    engine.runFullMachExperience();
}

also, the recreate part might come in handy in the future if we want to support switching adapters on the fly. Think on battery low request low-power adapter. Will also allow to change in game options without restart.

if we have only one entry point, "main on top of main" and give engine to that and leave user decide what api to use. manual low level way ``` pub fn my_main(engine: *mach.Engine) { engine.set_size(1024, 768) engine.set_vsync(.triple) engine.recreate() // just an idea to avoid magic Options const back_buffer = engine.startFrame(); // can hide some lowest level things like that while (engine.pollEvents()) { } doManualRender(); engine.swap() } ``` full mach ``` pub fn my_main(engine: *mach.Engine) { engine.set_size(1024, 768) engine.set_vsync(.triple) engine.recreate() // just an idea to avoid magic Options engine.setECSParams(); engine.setPhysicsParams(); engine.maybeSetSomeCallbacks(); engine.runFullMachExperience(); } ``` also, the `recreate` part might come in handy in the future if we want to support switching adapters on the fly. Think on battery low request low-power adapter. Will also allow to change in game options without restart.

iddev5 commented 2022-05-28 11:29:49 +00:00 (Migrated from github.com)

Copy link

@d3m1gd we cannot have only one entry point and have a while (...) loop inside it because wasm and Android doesn't allow that. That's the entire reason why we got custom entry points in the first place.

@d3m1gd we cannot have only one entry point and have a while (...) loop inside it because wasm and Android doesn't allow that. That's the entire reason why we got custom entry points in the first place.

d3m1gd commented 2022-05-29 03:17:56 +00:00 (Migrated from github.com)

Copy link

i see. so, back to the custom entry points then.

since init takes App, we can have another step setup as in:

pub fn machSetup(allocator: std.mem.Allocator) *App {
    var app = allocator.create(App);
    app.options = mach.Options{};
    app.otherInit();
    return app;
}

pub fn init(app: *App, engine: *mach.Engine) {
}

this allows to fill Options without magic, and is exactly apparent where App comes from, as allocator implies we must allocate it ourselves.

This way there is exactly one mental step to burden users with: "define needed functions with proper signatures", instead of additional "define pub const Options".

If recreating engine live is an option, we could even extend machSetup to take oldApp, as in:

pub fn machSetup(allocator: std.mem.Allocator, oldApp: ?*App) *App {
    if (oldApp) |old| {
        // recreation, maybe user toggled vsync
        defer old.deinit(); // user defined method on App
        var app = allocator.create(App);
        app.reusePartsOfOld(old);
        return app;
    }

   // as before, normal creation from scratch
}

this way user could just recreate engine as a request

pub fn update(app: *App, engine: *Engine) {
    if (user_wants_to_toggle_options) {
        app.new_options = .{ a, b, c };
        engine.please_recreate(); // sets flag internally, will call machSetup with app as oldApp
        return;
    }
}

i see. so, back to the custom entry points then. since init takes App, we can have another step `setup` as in: ``` pub fn machSetup(allocator: std.mem.Allocator) *App { var app = allocator.create(App); app.options = mach.Options{}; app.otherInit(); return app; } pub fn init(app: *App, engine: *mach.Engine) { } ``` this allows to fill Options without magic, and is exactly apparent where App comes from, as allocator implies we must allocate it ourselves. This way there is exactly one mental step to burden users with: "define needed functions with proper signatures", instead of additional "define pub const Options". If recreating engine live is an option, we could even extend `machSetup` to take oldApp, as in: ``` pub fn machSetup(allocator: std.mem.Allocator, oldApp: ?*App) *App { if (oldApp) |old| { // recreation, maybe user toggled vsync defer old.deinit(); // user defined method on App var app = allocator.create(App); app.reusePartsOfOld(old); return app; } // as before, normal creation from scratch } ``` this way user could just recreate engine as a request ``` pub fn update(app: *App, engine: *Engine) { if (user_wants_to_toggle_options) { app.new_options = .{ a, b, c }; engine.please_recreate(); // sets flag internally, will call machSetup with app as oldApp return; } } ```

d3m1gd commented 2022-05-29 03:22:18 +00:00 (Migrated from github.com)

Copy link

on the point of deinit being optional, it doesn't have to be, but if user doesn't want to deinit anything it will just be an empty function serving no real purpose. There is no real technical reason for it to be optional, so it can stay as non-optional, no problem. But overall, we could put such methods as callbacks onto engine itself if needed.

pub fn init(app: *App, engine: *mach.Engine) {
     engine.app_deinit_callback(...); 
}

on the point of deinit being optional, it doesn't have to be, but if user doesn't want to deinit anything it will just be an empty function serving no real purpose. There is no real technical reason for it to be optional, so it can stay as non-optional, no problem. But overall, we could put such methods as callbacks onto engine itself if needed. ``` pub fn init(app: *App, engine: *mach.Engine) { engine.app_deinit_callback(...); } ```

iddev5 commented 2022-05-29 05:57:38 +00:00 (Migrated from github.com)

Copy link

While I like the idea of having a setup() function, there are some problems with above:

• App is currently allocated on stack. Going forward we might always want to allocate it on heap. (Reason below)
• App is just a struct holding game/app state. It's not a functioning part itself. It doesn't have any predefined function itself. The file you're working on when making a game/app is App itself. Since there might be lots of game state in an application, it would be better to always heap allocate it and not giving the user a chance to accidentally overflow their stack.

So I suggest that setup() should just return a mach.Options, and not an App. It's also better if we could keep it's name to just setup() instead of machSetup()

---

If recreating engine live is an option

Recreating engine doesn't need modifications to App. So setup taking oldApp is not needed.

What we can do to facilate recreation is to have a recreate() function in engine taking a mach.Option.
NOTE: we can use this function in init to create the context but init already takes an Engine which is initialized. This means that very closely we are creating-destroying-creating the Engine for no reason. If we keep the initial engine uninitialized, then calling the recreate() function (but this time with a different name like just create()) at the very beginning of init is necessary, otherwise it would let to strange uniitialized behaviors.

---

Since there has been enough confusion on what App is and we are already considering renaming it, we should rename App to State. (Name suggestion)

---

on the point of deinit being optional

See above. We actually went for two routes

1. init() is the only necessary function and all state is kept inside it so deinit is not needed at all. We can (and we should) remove the App param from this function. It can optionally be renamed to something less confusing. This is emulating a traditional entry point
2. init() and deinit() both are necessary. App parameter is needed.

I don't like the approach of setting deinit as a callback by a function because if user made an init function they would naturally expect to be able to create a deinit function.

While I like the idea of having a ``setup()`` function, there are some problems with above: - App is currently allocated on stack. Going forward we might always want to allocate it on heap. (Reason below) - App is just a struct holding game/app state. It's not a functioning part itself. It doesn't have any predefined function itself. The file you're working on when making a game/app is App itself. Since there might be lots of game state in an application, it would be better to always heap allocate it and not giving the user a chance to accidentally overflow their stack. So I suggest that setup() should just return a mach.Options, and not an App. It's also better if we could keep it's name to just setup() instead of machSetup() --- > If recreating engine live is an option Recreating engine doesn't need modifications to App. So setup taking oldApp is not needed. What we can do to facilate recreation is to have a recreate() function in engine taking a mach.Option. NOTE: we can use this function in init to create the context but init already takes an Engine which is initialized. This means that very closely we are creating-destroying-creating the Engine for no reason. If we keep the initial engine uninitialized, then calling the recreate() function (but this time with a different name like just create()) at the very beginning of init is necessary, otherwise it would let to strange uniitialized behaviors. --- Since there has been enough confusion on what App is and we are already considering renaming it, we should rename App to ``State``. (Name suggestion) --- > on the point of deinit being optional See above. We actually went for two routes 1) init() is the only necessary function and all state is kept inside it so deinit is not needed at all. We can (and we should) remove the App param from this function. It can optionally be renamed to something less confusing. This is emulating a traditional entry point 2) init() and deinit() both are necessary. App parameter is needed. I don't like the approach of setting deinit as a callback by a function because if user made an init function they would naturally expect to be able to create a deinit function.

d3m1gd commented 2022-05-29 06:25:57 +00:00 (Migrated from github.com)

Copy link

so lets just pass engine uninitialized to init, allowing user to engine.init(Options), and add checks in every user facing api function to verify they are called after engine.init with some flag in engine, but only on dev builds, so those checks wont affect release performance. Relevant error message (as well as documentation) will inform user of necessity of engine.init

so lets just pass engine uninitialized to init, allowing user to `engine.init(Options)`, and add checks in every user facing api function to verify they are called after `engine.init` with some flag in engine, *but* only on dev builds, so those checks wont affect release performance. Relevant error message (as well as documentation) will inform user of necessity of `engine.init`

d3m1gd commented 2022-05-29 06:30:06 +00:00 (Migrated from github.com)

Copy link

so far whatever name we choose App or State will force user to do pub const State = @This();

if there were a way to iterate over all decls in user entry source file (which we know in build.zig App.init), we could find the fn init(xxx: XXX, engine: Engine) and grab XXX type and user can choose whatever name they like.

so far whatever name we choose `App` or `State` will force user to do `pub const State = @This();` if there were a way to iterate over all decls in user entry source file (which we know in build.zig App.init), we could find the `fn init(xxx: XXX, engine: Engine)` and grab `XXX` type and user can choose whatever name they like.

iddev5 commented 2022-05-29 09:10:11 +00:00 (Migrated from github.com)

Copy link

so lets just pass engine uninitialized to init

I do remember the glfw bindings implement similar checks regarding glfw.init(). Let's see slimsag's opinion in this.

if there were a way to iterate over all decls in

This is not needed. the user can do const Whatever = @This();

> so lets just pass engine uninitialized to init I do remember the glfw bindings implement similar checks regarding glfw.init(). Let's see slimsag's opinion in this. > if there were a way to iterate over all decls in This is not needed. the user can do ``const Whatever = @This();``

alichraghi commented 2022-05-29 10:20:00 +00:00 (Migrated from github.com)

Copy link

first of all we must know what we are gonna write. a library or a framework and toolchain like unity, ue, godot, etc ?

a Library

for some developers this may be preferable because mach apps will be more portable/modular, developer has more control and can use mach as an embeddable module. this means we lose the control to provide wasm target any many more. so developer should provide it's own implemntation

by making mach a library we will need to completely change current function/struct passing API. this has some benefits. expect a mid-level framework that just introduce more problems like custom context/unsafe pointer casts/etc app is the whole context not a part of context.

my windowing entry point

pub fn main() void {
  var app = mach.init(options);
  defer app.deinit();
  app.createWindow(800, 640, "hello");
  while (engine.pollEvents()) |_| {
    switch (event) {
      .key_press => |ev| {
        if (ev.key == .space)
          ...
        },
        else => {},
    }
    ...
    app.drawText("xyz");
    engine.gpu_driver.swap_chain.?.present();
  }
}

my wasm entry point

pub fn my_wasm_main() void {
  ...
}

a Framework

this way also has some benfits:

• more control and optimizations by mach
• easier usage
• GUI app for level editor, code editor, etc

since we don't provide a fully high-level API the only remaining way i can though is the current App -> init(), update(), deinit() API.

first of all we must know what we are gonna write. a **library** or a **framework and toolchain** like unity, ue, godot, etc ? ## a Library for some developers this may be preferable because mach apps will be more portable/modular, developer has more control and can use mach as an embeddable module. this means we lose the control to provide wasm target any many more. so developer should provide it's own implemntation by making mach a library we will need to completely change current function/struct passing API. this has some benefits. expect a mid-level framework that just introduce more problems like custom context/unsafe pointer casts/etc app is the whole context not a part of context. #### my windowing entry point ```zig pub fn main() void { var app = mach.init(options); defer app.deinit(); app.createWindow(800, 640, "hello"); while (engine.pollEvents()) |_| { switch (event) { .key_press => |ev| { if (ev.key == .space) ... }, else => {}, } ... app.drawText("xyz"); engine.gpu_driver.swap_chain.?.present(); } } ``` #### my wasm entry point ```zig pub fn my_wasm_main() void { ... } ``` ## a Framework this way also has some benfits: - more control and optimizations by mach - easier usage - GUI app for level editor, code editor, etc since we don't provide a fully high-level API the only remaining way i can though is the current `App -> init(), update(), deinit()` API.

iddev5 commented 2022-05-29 10:37:26 +00:00 (Migrated from github.com)

Copy link

I would say it's "framework" according to your definition. We want to provide a cross platform easy to use engine for developing apps without worrying about special cases as much as possible.

The "library" approach lacks all the above things. Moreover mach already provided glfw, gpu, freetype, ecs etc libraries to use as per you want. And if needed in future we can convert the wasm code into its own mach-wasm module.

I would say it's "framework" according to your definition. We want to provide a cross platform easy to use engine for developing apps without worrying about special cases as much as possible. The "library" approach lacks all the above things. Moreover mach already provided glfw, gpu, freetype, ecs etc libraries to use as per you want. And if needed in future we can convert the wasm code into its own mach-wasm module.

👍 1

emidoots commented 2022-05-29 12:58:20 +00:00 (Migrated from github.com)

Copy link

I think @d3m1gd's concerns were mostly addressed by clarifying: regardless of whether we have Engine, Core, or both - neither of those should be aware of physics, gui, etc. in any capacity.

Based on the chat in Matrix, I think we have agreement on the following:

---

A high-level app would look something like:

pub fn init() {
    engine.ecs.addModule(mach.Physics);
}

Or perhaps

pub fn init() {
    mach.addStandardModules(engine);
}

---

Two remaining questions were "whether or not this unified thing is called "Core" or "Engine", as well as "can I get window/input/webgpu without ECS at all?""

Conclusion: call it Engine and say no, you always get ECS, but you're free to just not use it (config option.) It's small anyway, and if you don't use it (we can just not call ecs.init for example) it uses no memory due to dead code elimination.

I think @d3m1gd's concerns were mostly addressed by clarifying: regardless of whether we have Engine, Core, or both - neither of those should be aware of physics, gui, etc. in any capacity. Based on the chat in Matrix, I think we have agreement on the following: --- A high-level app would look something like: ```zig pub fn init() { engine.ecs.addModule(mach.Physics); } ``` Or perhaps ```zig pub fn init() { mach.addStandardModules(engine); } ``` --- Two remaining questions were "whether or not this unified thing is called "Core" or "Engine", as well as "can I get window/input/webgpu without ECS at all?"" Conclusion: call it `Engine` and say no, you always get ECS, but you're free to just not use it (config option.) It's small anyway, and if you don't use it (we can just not call `ecs.init` for example) it uses no memory due to dead code elimination.

emidoots commented 2022-05-29 13:08:57 +00:00 (Migrated from github.com)

Copy link

@iddev5

EDIT: SOLUTION: We can define run() as following:

Nice, I like this a lot 👍

Since there has been enough confusion on what App is and we are already considering renaming it, we should rename App to State. (Name suggestion)

Honestly, I'm not really sure if I like this better or if I just think it's different. Anyone else have feedback on this?

This is not needed. the user can do const Whatever = @This();

I agree, inspecting file declarations as an alternative to this sounds even more magical - not what I want.

App is currently allocated on stack. Going forward we might always want to allocate it on heap. (Reason below)

Tentatively I agree with this, it avoids a serious footgun so let's do it. What might change it is if Zig in the future gets lots of safety features around stack memory misuse.

I do remember the glfw bindings implement similar checks regarding glfw.init(). Let's see slimsag's opinion in this.

@iddev5 can you clarify your thoughts around setup()? I'm having trouble piecing together from this thread how you envision setup() could play a role here. Also happy to just say 'do what feels right to you' here, I think you've heard enough of my POV at this point to know which direction we should go in generally here.

@iddev5 > EDIT: SOLUTION: We can define run() as following: Nice, I like this a lot 👍 > Since there has been enough confusion on what App is and we are already considering renaming it, we should rename App to State. (Name suggestion) Honestly, I'm not really sure if I like this better or if I just think it's different. Anyone else have feedback on this? > This is not needed. the user can do `const Whatever = @This();` I agree, inspecting file declarations as an alternative to this sounds even more magical - not what I want. > App is currently allocated on stack. Going forward we might always want to allocate it on heap. (Reason below) Tentatively I agree with this, it avoids a serious footgun so let's do it. What might change it is if Zig in the future gets lots of safety features around stack memory misuse. > I do remember the glfw bindings implement similar checks regarding `glfw.init()`. Let's see slimsag's opinion in this. @iddev5 can you clarify your thoughts around `setup()`? I'm having trouble piecing together from this thread how you envision `setup()` could play a role here. Also happy to just say 'do what feels right to you' here, I think you've heard enough of my POV at this point to know which direction we should go in generally here.

emidoots commented 2022-05-29 13:20:25 +00:00 (Migrated from github.com)

Copy link

@alichraghi I would classify Mach as somewhere between your "framework" and "toolchain like unity, ue, godot" - it's a bit hard to explain, but here goes:

I think "framework" heavily implies that you're not getting a one-true-editor experience (the experience you get with unity/ue/godot.) Maybe similar to say raylib, you get excellent standalone tools that you can use with the framework, but not really a unified experience across all of them necessarily.

On the opposite end, I think "toolchain like unity/ue/godot" heavily implies "clunky, large, non-composable" - you do things their way generally, and your application must fit into the unity/ue/godot model. It's not generally supported to say "I want a different GUI, physics engine, renderer, etc." and be able to swap those out in a serious way.

I don't think we need more frameworks in the world, and I don't think we need more unity/ue/godots in the world. What we do need is a better middleground: something that is at least in spirit competitive with unity/ue, because many great games are complex beyond what frameworks can handle, yet composable like a framework. This is only achievable if (1) you have a truly good graphics abstraction everyone can agree on (WebGPU is our golden goose) and (2) you have a truly good interoperability abstraction everyone can agree on (it's my hope ECS will do this for us.) Then you can add a one-true-editor on top which heavily pushes the idea of data-oriented tooling (produce data, like a 3D editor / image editor does, which you consume in your app), a bit of runtime debug tooling, and we're good to go.

@alichraghi I would classify Mach as somewhere between your "framework" and "toolchain like unity, ue, godot" - it's a bit hard to explain, but here goes: I think "framework" heavily implies that you're not getting a one-true-editor experience (the experience you get with unity/ue/godot.) Maybe similar to say raylib, you get excellent standalone tools that you can use with the framework, but not really a unified experience across all of them necessarily. On the opposite end, I think "toolchain like unity/ue/godot" heavily implies "clunky, large, non-composable" - you do things their way generally, and your application must fit into the unity/ue/godot model. It's not generally supported to say "I want a different GUI, physics engine, renderer, etc." and be able to swap those out in a serious way. I don't think we need more frameworks in the world, and I don't think we need more unity/ue/godots in the world. What we do need is a better middleground: something that is at least in spirit competitive with unity/ue, because many great games _are_ complex beyond what frameworks can handle, yet composable like a framework. This is only achievable if (1) you have a truly good graphics abstraction everyone can agree on (WebGPU is our golden goose) and (2) you have a truly good interoperability abstraction everyone can agree on (it's my hope ECS will do this for us.) Then you can add a one-true-editor on top which heavily pushes the idea of data-oriented tooling (produce data, like a 3D editor / image editor does, which you consume in your app), a bit of runtime debug tooling, and we're good to go.

d3m1gd commented 2022-05-29 13:20:29 +00:00 (Migrated from github.com)

Copy link

@iddev5 can you clarify your thoughts around setup()? I'm having trouble piecing together from this thread how you envision setup() could play a role here. Also happy to just say 'do what feels right to you' here, I think you've heard enough of my POV at this point to know which direction we should go in generally here.

We need to decide on how to init Engine with Options. The latest idea is to pass Engine allocated but undefined, and let user to call init on it with options.

pub fn init(app: *App, engine: *Engine) !void {
    // engine undefined here
    engine.init(options) // mach.Options
    app.doStuffWithEngine(engine); // legit here, would error before init above
}

Tentatively I agree with this, it avoids a serious footgun so let's do it. What might change it is if Zig in the future gets lots of safety features around stack memory misuse.

since we control now how to allocate App we can change as we go back and forth, it is hidden from the user anyway

> @iddev5 can you clarify your thoughts around setup()? I'm having trouble piecing together from this thread how you envision setup() could play a role here. Also happy to just say 'do what feels right to you' here, I think you've heard enough of my POV at this point to know which direction we should go in generally here. We need to decide on how to init `Engine` with `Options`. The latest idea is to pass `Engine` allocated but undefined, and let user to call `init` on it with options. ``` pub fn init(app: *App, engine: *Engine) !void { // engine undefined here engine.init(options) // mach.Options app.doStuffWithEngine(engine); // legit here, would error before init above } ``` > Tentatively I agree with this, it avoids a serious footgun so let's do it. What might change it is if Zig in the future gets lots of safety features around stack memory misuse. since we control now how to allocate `App` we can change as we go back and forth, it is hidden from the user anyway

iddev5 commented 2022-05-29 13:23:04 +00:00 (Migrated from github.com)

Copy link

@slimsag Basically what d3m1gd mentioned above. The other option is to have user define a function called setup alongside init and all. The setup() function will return a mach.Options. With this option mach.Engine arg in init is always defined and initialized.

With the first option explained above, the user has to initialize it themselves.

@d3m1gd i did noticed that its as simple as adding a safety check in all functions since engine has fields like device whose methods arent defined inside mach.

@slimsag Basically what d3m1gd mentioned above. The other option is to have user define a function called setup alongside init and all. The setup() function will return a mach.Options. With this option mach.Engine arg in init is always defined and initialized. With the first option explained above, the user has to initialize it themselves. @d3m1gd i did noticed that its as simple as adding a safety check in all functions since engine has fields like device whose methods arent defined inside mach.

emidoots commented 2022-05-29 16:12:47 +00:00 (Migrated from github.com)

Copy link

Concrete proposal

• Mach owns the entrypoint main (it has to, in wasm main function cannot block.)
• Instead, we call your main.zig's pub fn init to give you an experience somewhat like have a main function.

Configuration

There are three types of options one can configure Mach with:

• comptime-only options, always configured via build.zig in the App struct. e.g. Android app title.
• startup-only options: things you usually cannot change once the app has started - these don't really exist on Android/iOS/WASM, and may be rare/obscure on Desktop. Until our code improves, though, it might include things like e.g. which GPU to use, opengl/vulkan switch, whether you want a high or low-performance GPU and glfw.Init options.
• runtime options: (think: whether vsync is on/off, window title/size, etc.) - can generally change at any point.

High-level applications

A high-level application (most Mach users) looks like this:

pub fn init(engine: *Engine) !void {
    // engine is initialized and configured for you already.

    // You can request runtime Options change using this:
    engine.setOptions(mach.Options{});

    // Equal to this basically:
    // engine.addModule(mach.renderer.plugin);
    // engine.addModule(mach.physics.plugin);
    engine.addModules(mach.standard_plugins);

    // myApp is an ECS system / module, which is where your app state can live.
    engine.addModule(myApp);
}

• If you choose to write a high-level application only init is available.
• Application state is stored in the ECS (via startup systems, etc.) I won't go into detail about this here, just saying this is how you'll do it.
• runtime options can be specified via engine.setOptions inside your init.

Startup-only options

If not specified in hard-coded list, defaults are used. Can override with env vars, e.g. GPU_BACKEND=vulkan.

App{
    .startup_options = .{
        .backend = .opengl,
    },
}

Here startup_options would correspond to a new struct type mach.StartupOptions, which may be empty for now.

Supporting runtime changes of things like power_preference, or required_limits, in the current mach.Options struct may be a bit involved - but we can just leave them in Options as-is and document that it's a bug they cannot be changed at runtime for now

Low-level applications

As mentioned earlier, supporting this is an explicit goal:

"I just want a cross-platform window/input basically, I'll use WebGPU directly for rendering and pretty much nothing else Mach provides"

In this mode, you may still technically have the ECS functions available to you, but they have effectively no impact & will end up mostly eliminated by dead code elimination.

In this mode, your main.zig must look like:

pub const App = @This();

pub fn init(app: *App, engine: *Engine) !void {
    // app has been heap allocated for you to zero value App{}.
    // engine is initialized using your startup() options.

    // You can request runtime Options change using this:
    engine.setOptions(mach.Options);

    // You can use engine.gpu here.
}

pub fn deinit(app: *App, engine: *mach.Engine) void {
    // Called on exit.
}

pub fn update(app: *App, engine: *Engine) !bool {
    // Called once per frame.
}

pub fn resize(app: *App, engine: *mach.Engine, width: u32, height: u32) !void {
    // ...
}

Each of these are required function signatures, enforced by @hasField and other comptime type checking.

# Concrete proposal * Mach owns the entrypoint `main` (it has to, in wasm main function cannot block.) * Instead, we call your `main.zig`'s `pub fn init` to give you an experience somewhat like have a `main` function. ## Configuration There are three types of options one can configure Mach with: * **comptime-only options**, always configured via `build.zig` in the `App` struct. e.g. Android app title. * **startup-only options**: things you usually cannot change once the app has started - these don't really exist on Android/iOS/WASM, and may be rare/obscure on Desktop. Until our code improves, though, it might include things like e.g. which GPU to use, opengl/vulkan switch, whether you want a high or low-performance GPU and `glfw.Init` options. * **runtime options**: (think: whether vsync is on/off, window title/size, etc.) - can generally change at any point. # High-level applications A high-level application (most Mach users) looks like this: ```zig pub fn init(engine: *Engine) !void { // engine is initialized and configured for you already. // You can request runtime Options change using this: engine.setOptions(mach.Options{}); // Equal to this basically: // engine.addModule(mach.renderer.plugin); // engine.addModule(mach.physics.plugin); engine.addModules(mach.standard_plugins); // myApp is an ECS system / module, which is where your app state can live. engine.addModule(myApp); } ``` * If you choose to write a high-level application **only `init` is available.** * Application state is stored in the ECS (via startup systems, etc.) I won't go into detail about this here, just saying this is how you'll do it. * **runtime options** can be specified via `engine.setOptions` inside your `init`. ## Startup-only options If not specified in hard-coded list, defaults are used. Can override with env vars, e.g. `GPU_BACKEND=vulkan`. ```zig App{ .startup_options = .{ .backend = .opengl, }, } ``` Here `startup_options` would correspond to a new struct type `mach.StartupOptions`, which may be empty for now. Supporting runtime changes of things like `power_preference`, or `required_limits`, in the current `mach.Options` struct may be a bit involved - but we can just leave them in `Options` as-is and document that it's a bug they cannot be changed at runtime for now ## Low-level applications As mentioned earlier, supporting this is an explicit goal: > "I just want a cross-platform window/input basically, I'll use WebGPU directly for rendering and pretty much nothing else Mach provides" In this mode, you may still technically have the ECS functions available to you, but they have effectively no impact & will end up mostly eliminated by dead code elimination. In this mode, your `main.zig` must look like: ```zig pub const App = @This(); pub fn init(app: *App, engine: *Engine) !void { // app has been heap allocated for you to zero value App{}. // engine is initialized using your startup() options. // You can request runtime Options change using this: engine.setOptions(mach.Options); // You can use engine.gpu here. } pub fn deinit(app: *App, engine: *mach.Engine) void { // Called on exit. } pub fn update(app: *App, engine: *Engine) !bool { // Called once per frame. } pub fn resize(app: *App, engine: *mach.Engine, width: u32, height: u32) !void { // ... } ``` Each of these are **required** function signatures, enforced by `@hasField` and other comptime type checking.

d3m1gd commented 2022-05-30 01:28:09 +00:00 (Migrated from github.com)

Copy link

What if we make setup an only non-optional function?

// please note, naming is not important here, just the concept

this solution unifies many ideas discussed:

1. eliminates the need for App parameter for high-level mach
2. eliminates the need to check for optional functions
3. requires high level users only 2 lines of manual top level definitions (@This() + setup)
4. explicitly requests low-level mode for low-level users
5. overall a familiar pattern to many

this is all that is needed for high level mach app.

pub const App = @This();

pub fn setup() mach.Engine.Builder {
  var builder = mach.Engine.builder(options); // startup + runtime options
  builder.add_startup_system(add_monsters);
  builder.add_system(renderer);
  builder.add_system(physics);
  return builder;
}

low-level mach must be explicitly required. which will make mach look for update init deinit

pub const App = @This();

pub fn setup() mach.Engine.Builder {
  var builder = mach.Engine.builder(options);
  builder.i_want_low_level(); // default internal flag is to be high-level, this changes it to low level mach
  return builder;
}

mach internals

pub fn main() { // not user code, mach pseudo internals for native
  var builder = App.setup();
  var engine = builder.makeEngine();
  if (engine.high_level) {
    while(true) {
         engine.update();
    }
  } else {
    checkHasProperDecls(App); // init + update + deinit
    App.init(app, engine);
    while(true) {
       App.update(app, engine);
    }
  }
}

What if we make `setup` an only non-optional function? // please note, naming is not important here, just the concept this solution unifies many ideas discussed: 1. eliminates the need for App parameter for high-level mach 2. eliminates the need to check for optional functions 3. requires high level users only 2 lines of manual top level definitions (`@This()` + `setup`) 4. explicitly requests low-level mode for low-level users 5. overall a familiar pattern to many this is all that is needed for high level mach app. ``` pub const App = @This(); pub fn setup() mach.Engine.Builder { var builder = mach.Engine.builder(options); // startup + runtime options builder.add_startup_system(add_monsters); builder.add_system(renderer); builder.add_system(physics); return builder; } ``` low-level mach must be explicitly required. which will make mach look for `update` `init` `deinit` ``` pub const App = @This(); pub fn setup() mach.Engine.Builder { var builder = mach.Engine.builder(options); builder.i_want_low_level(); // default internal flag is to be high-level, this changes it to low level mach return builder; } ``` mach internals ``` pub fn main() { // not user code, mach pseudo internals for native var builder = App.setup(); var engine = builder.makeEngine(); if (engine.high_level) { while(true) { engine.update(); } } else { checkHasProperDecls(App); // init + update + deinit App.init(app, engine); while(true) { App.update(app, engine); } } } ```

emidoots commented 2022-05-30 02:29:07 +00:00 (Migrated from github.com)

Copy link

After chatting with @d3m1gd we agree on my original proposal above, with a slight clarification:

• If pub fn init(engine: *Engine) !void is present -> it's a high-level app, type signature verified at comptime
• If no init present -> helpful comptime error message saying to declare high-level init function

Only if you declare .low_level = true in build.zig App config can you use the low-level API. It then enforces you have the right function signatures at comptime, including:

pub fn init(app: *App, engine: *Engine) !void

After chatting with @d3m1gd we agree on [my original proposal above](https://github.com/hexops/mach/issues/309#issuecomment-1140479760), with a slight clarification: * If `pub fn init(engine: *Engine) !void` is present -> it's a high-level app, type signature verified at comptime * If no `init` present -> helpful comptime error message saying to declare high-level `init` function Only if you declare `.low_level = true` in `build.zig` `App` config can you use the low-level API. It then enforces you have the right function signatures at comptime, including: ``` pub fn init(app: *App, engine: *Engine) !void ```

👍 1

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

sg/entrypoint

wrench/update-zig

sg/module-system-v2

sg/linux-stub

0.4

core-api

sysmidi

sg/perspective

v0.4.0

v0.3.0

v0.2.0

v0.1

Labels

Clear labels   

CI

  

all

  

basisu

  

blog

  

bug

Something isn't working

  

build

  

contributor-friendly

Good issue for contributors to take on

  

core

  

correctness

  

deferred

  

dev

  

direct3d-headers

  

docs

Improvements or additions to documentation

  

driver-os-issue

Issues that are very likely to be graphics driver / OS configuration issues

  

duplicate

This issue or pull request already exists

  

dxcompiler

  

editor

Graphical editor and Mach CLI

  

examples

  

experiment

  

feature-idea

New feature or request

  

feedback

  

flac

  

freetype

  

gamemode

  

gkurve

  

glfw

  

gpu

  

gpu-dawn

  

harfbuzz

  

help welcome

  

in-progress

  

infrastructure

  

invalid

This doesn't seem right

  

libmach

  

linux-audio-headers

  

long-term

  

mach

  

mach.gfx

  

mach.math

  

mach.physics

  

mach.testing

  

model3d

  

needs-triage

Issues that need triaging

  

object

object system

  

opengl-headers

  

opus

  

os/linux

  

os/macos

  

os/wasm

WebAssembly support

  

os/windows

  

package-manager

  

priority

  

proposal

  

proposal-accepted

  

question

Further information is requested

  

roadmap

  

slipped

  

stability

  

sysaudio

  

sysgpu

  

sysjs

  

validating-fix

  

vulkan-zig-generated

  

wayland-headers

  

website

  

wontfix

This will not be worked on

  

wrench

  

www

  

x11-headers

  

xcode-frameworks

  

zig-update

No labels

CI

all

basisu

blog

bug

build

contributor-friendly

core

correctness

deferred

dev

direct3d-headers

docs

driver-os-issue

duplicate

dxcompiler

editor

examples

experiment

feature-idea

feedback

flac

freetype

gamemode

gkurve

glfw

gpu

gpu-dawn

harfbuzz

help welcome

in-progress

infrastructure

invalid

libmach

linux-audio-headers

long-term

mach

mach.gfx

mach.math

mach.physics

mach.testing

model3d

needs-triage

object

opengl-headers

opus

os/linux

os/macos

os/wasm

os/windows

package-manager

priority

proposal

proposal-accepted

question

roadmap

slipped

stability

sysaudio

sysgpu

sysjs

validating-fix

vulkan-zig-generated

wayland-headers

website

wontfix

wrench

www

x11-headers

xcode-frameworks

zig-update

Milestone

Clear milestone

No items

No milestone

Mach 0.2

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

hexops/mach#309

No description provided.