ecs: type-safe queries/iteration #402

Merged

dweiller merged 2 commits from typed-query into main

2022-07-15 04:49:20 +00:00

dweiller commented

2022-07-11 09:14:10 +00:00

(Migrated from github.com)

How should the api for type-safe queries and iteration look? In the future when multithreading/scheduling is worked on this will probably need revisiting as knowledge of what kinds of queries/iteration is done can be exploited for scheduling, but in the meantime it would be good to have a type-safe way to iterate.

By selecting this checkbox, I agree to license my contributions to this project under the license(s) described in the LICENSE file, and I have the right to do so or have received permission to do so by an employer or client I am producing work for whom has this right.

Now that we have typed ecs components, it makes sense to get type safety into the query/iteration api. This is a first past that works, but I'm not thrilled with the api (passing the query to the `next()` function rather than the `query()` function that constructs the `Iterator`. To make the components a comptime parameter to `query()` instead of `next()` requires the information about the query to be stored in the iterator, which would mean `Iterator` has to be generic over the components queried. Alternatively the component query could be runtime information, but that would require allocating (to concatenate namespace and component name), or imposed a maximum length for namespace + component name. How should the api for type-safe queries and iteration look? In the future when multithreading/scheduling is worked on this will probably need revisiting as knowledge of what kinds of queries/iteration is done can be exploited for scheduling, but in the meantime it would be good to have a type-safe way to iterate. - [x] By selecting this checkbox, I agree to license my contributions to this project under the license(s) described in the LICENSE file, and I have the right to do so or have received permission to do so by an employer or client I am producing work for whom has this right.

emidoots commented

2022-07-12 14:00:42 +00:00

(Migrated from github.com)

Thanks for this. I've yet to get around to looking at querying in depth yet, realistically I think that querying cannot be looked at in isolation. Here's what I mean:

Modules
Modules doing message-passing
Multi-threaded scheduling of system execution
Queries

These four items must be considered together, because each one impacts the other. So far I've gotten close to completing the first two, and investigating the third, but not yet all of them together.

I'm happy to merge this PR as-is right now, though, because it's a clear improvement over what we have today. Any objection to that?

Thanks for this. I've yet to get around to looking at querying in depth yet, realistically I think that querying cannot be looked at in isolation. Here's what I mean: * Modules * Modules doing message-passing * Multi-threaded scheduling of system execution * Queries These four items must be considered together, because each one impacts the other. So far I've gotten close to completing the first two, and investigating the third, but not yet all of them together. I'm happy to merge this PR as-is right now, though, because it's a clear improvement over what we have today. Any objection to that?

dweiller commented

2022-07-13 09:03:04 +00:00

(Migrated from github.com)

I've pushed a second commit that fixes API so that you pass the components you want to query to the query() function rather than next(), by making the Iterator type generic over those components.

The reason I made this PR a draft was essentially because of those other parts of Mach that querying needs to interact with and wasn't sure how much thought others have put into the API. I'm happy to have these changes merged - these changes can be a starting point for developing the API as those other aspects are considered/implemented (I guess it would be very hard to get all of it 'right' in one go anyway).

I'm not certain what you mean with queries and Modules (and message passing) impacting each other (unless all you mean is that the comptime known components must match up).

I've pushed a second commit that fixes API so that you pass the components you want to query to the `query()` function rather than `next()`, by making the `Iterator` type generic over those components. The reason I made this PR a draft was essentially because of those other parts of Mach that querying needs to interact with and wasn't sure how much thought others have put into the API. I'm happy to have these changes merged - these changes can be a starting point for developing the API as those other aspects are considered/implemented (I guess it would be very hard to get all of it 'right' in one go anyway). I'm not certain what you mean with queries and Modules (and message passing) impacting each other (unless all you mean is that the comptime known components must match up).

emidoots commented

2022-07-15 04:48:22 +00:00

(Migrated from github.com)

these changes can be a starting point for developing the API as those other aspects are considered/implemented (I guess it would be very hard to get all of it 'right' in one go anyway).

Yup, that's exactly what I mean :)

I'm not certain what you mean with queries and Modules (and message passing) impacting each other (unless all you mean is that the comptime known components must match up).

Some examples:

Should systems support async execution / make use of Zig async? Multi-threading impacts this.
How should message passing work in a multi-threaded context? And in a single-threaded context? Async impacts this.
How should querying and interacting with ECS data work across threads? Points 1 and 2 impact this, because it could open up a "let systems fight it out with mutexes, interact with anything in the entire ECS" approach rather than a strict "define what you'll interact with up front" approach.

All this to say, there's a lot of conflicting concerns to address - and as you mentioned they can't all be addressed at once. But it's important to address all of them before we say the API is stabilized at all.

> these changes can be a starting point for developing the API as those other aspects are considered/implemented (I guess it would be very hard to get all of it 'right' in one go anyway). Yup, that's exactly what I mean :) > I'm not certain what you mean with queries and Modules (and message passing) impacting each other (unless all you mean is that the comptime known components must match up). Some examples: 1. Should systems support async execution / make use of Zig async? Multi-threading impacts this. 2. How should message passing work in a multi-threaded context? And in a single-threaded context? Async impacts this. 3. How should querying and interacting with ECS data work across threads? Points 1 and 2 impact this, because it could open up a "let systems fight it out with mutexes, interact with anything in the entire ECS" approach rather than a strict "define what you'll interact with up front" approach. All this to say, there's a _lot_ of conflicting concerns to address - and as you mentioned they can't all be addressed at once. But it's important to address all of them before we say the API is stabilized at all.

emidoots (Migrated from github.com) approved these changes

2022-07-15 04:49:09 +00:00