I've been spending a bunch of time reading about continuations and microthreading recently, just trying to wrap my head around them and find areas where they might make sense, specifically in C# using the ISO CLI systems. Of course, the example that most people give is AI, like so the one provided by this article:
IEnumerable Patrol () { while (alive){ if (CanSeeTarget ()) { yield return Attack (); } else if (InReloadStation){ Signal signal = AnimateReload (); yield return signal; } else { MoveTowardsNextWayPoint (); yield return TimeSpan.FromSeconds (1); }; } yield break; }
Now, implementation details aside (I'm not a fan of the authors implementation of the scheduler), you can then have a scheduler for all of your microthreads that will run each method until it terminates. Run one (or more) of these schedulers in a background thread and everything's awesome right?
Well, I'm not so sure. I have questions about this myself. For example, I would think that this pattern is more useful in cases where you have a lot of things that need to be done in sequence, but potentially in other threads or for multiple frames. For example the "AnimateReload" state from the example. Well, what if that state looked more like this?
if (NeedsReload)
{
Signal coverFound = BeginFindCover();
yield return coverFound;
if (CoverPoint != null)
{
Signal atCover = MoveToCover(CoverPoint);
yield return atCover;
}
Signal reloadAnimation = AnimateReload();
yield return reloadAnimation;
}
This is a more common use case in my mind. Have a procedure that takes many frames of execution and has portions that can be delegated to background tasks (such as path finding, which might take a few frames) and have them written linearly so that it's easier to read.
Here's the problem. Because we've now put the "Alive" check farther and farther away from the actual call, we've ended up in a situation where, at any time, the AI can die, and thus its microthread will need to die with it. To combat this, we'd have to check if the AI is actually alive after every yield, creating this code:
if (NeedsReload)
{
Signal coverFound = BeginFindCover();
yield return coverFound;
if (!Alive)
yield break;
if (CoverPoint != null)
{
Signal atCover = MoveToCover(CoverPoint);
yield return atCover;
if (!Alive)
yield break;
}
Signal reloadAnimation = AnimateReload();
yield return reloadAnimation;
}
To say nothing of if the state of the AI was forcibly changed for whatever reason (say, for example, if a Team AI Manager has told him to start running away, or regroup, etc). The only way I can think to combat this is through a special AIYield function which would handle most of that, but even then, I find it hard to think of a way to tell the system it needs to break from the current update thread and start a new one. Reschedule?
What do others think? Am I over thinking this? Am I thinking about this the wrong way? Am I using the wrong tool?
You’ve described a problem with co-routines, not continuations. Continuations can be used to implement co-routines (in languages that support continuations of course), but that’s not exactly what they’re for.
I’ve run into the same problem with Continuations (within the context of Unity.) One solution to the “Alive” problem is stopping any continuations running on the current script with a call like StopCoroutines() (which exists in Unity) upon the death of the AI. Still, it doesn’t fully solve the overall problem of irregular interruptions to the regular flow of the state diagram.
its better to do this check
if (!Alive)
yield break;
a part of action itself not the part of a sequence
In that case action will break sequence if something goes wrong according to action’s logic.
Or even get rid of enumeration style and do pure continuation as explained here “Stackless C# – Trampolining, Continuations and Erlang in .Net” http://refractalize.blogspot.com/2009/02/stackless-c-trampolining-continuations_21.html
@jdinolt Ah, i think I see the difference, but my question would still be, how do you solve the problem in co-routines? And is it necessarily bad to implement co-routines this way?
@Yilmaz Unfortunately the MS CLR has no concept of actual co-routines and ownership. You have to hold on to the enumerator which is essentially the stack information.
@Tomat Not sure Trampolining solves my problem.
While I have no experience with continuations or coroutines in C#, I can say that in general, I believe that you only keep calling back into the coroutine so long as its invariants hold. Another way to put it is that we don’t expect coroutines to run to completion and terminate; instead, as soon as the actor has died, you need to stop calling the UpdateAlive coroutine. Actions that depend on a live actor could then throw an exception if you try to MoveToCover with a dead actor, for example, but I probably wouldn’t rely on catching such cases as normal flow control (reserving that exception for truly exceptional cases).
Since we’re specifically talking about using coroutines in a separate thread to execute actions that may take place over several frames (and communicate back to other threads), there are all the usual issues of making sure the invariants truly stay invariant while a function (or in this case coroutine) is running, but between yields, presumably we can get updates that would terminate (or put on hold) coroutines which are no longer valid, such as “this monster just died.”
I get the feeling I might be misunderstanding the problem. I’m by no means an expert with this stuff, but I hope that helps.
I have some experience with Lua-based coroutines/microthreads. Usually, they end up getting attached to a single, parent entity. When the parent is removed from the game world, the coroutine is paused, and if the parent is deleted, the coroutine is deleted. This tends to work out in a surprising number of cases, although I’m usually using coroutines to write simple, top-to-bottom scripted animations, not AI. In cases where things get more complicated, I usually end up writing some kind of higher-level, game-specific controller to oversee synchronization.
Generator coroutines are only an approximation of microthreads built on an approximation of coroutines built on a programming language primitive that has been extended on a whim in some vaguely suitable fashion. They are ugly, and using them is ugly. Just look at “yield return”, it is not natural and is nonsensical to read.
Take a look a Python. First generators were added, and then someone said, “Hey, look, if we allow values to be sent through the generator to the yielding function, we get coroutines!” Then someone said, “If we write a scheduler that shoves these coroutines around, and the code written in the coroutines is written in a certain way with the custom boilerplate that allows these coroutines to work, then we have microthreading!” What you really have, is a situation where you have approximated microthreads that do not have the benefits of proper microthreads. Whether the creators of the generator coroutines and microthreading framework were ever aware of these benefits, is something that could be questioned.
So, what’s my point? Firstly that proper coroutines should make code more readable, not less readable. Secondly, that with a proper microthreading framework you do not need to add these checks in your logic. I am not familiar with the generator coroutine / microthreading implementation you are using, so you will have to work out how good an approximation it is.
Yes, the checks are cumbersome, but I had game logic that is probably still out there pumping away at the moment doing many steps in a microthread along the lines of “Approach target.. check if in process of being garbage collected, check if dead, check if still on that target, .. Still focused and alive? Proceed with next step of the action on that target.” The only reason these checks are still in place, is not because I could not abstract them away completely, but rather that the microthreading framework I used was not finalised at the time I initially wrote the logic. It would have been trivial to replace them after it was, if I had deemed worthwhile. The truth is that there really wasn’t that much entity logic that needed checks like this, most of it was pretty generic, including that generic action performing function that got launched by each entity.
So how would I have gotten rid of these checks on our microthreading framework? Well in Stackless Python you can kill a microthread. This reschedules the microthread but rather than continuing execution where it was yielded, instead an exception (TaskletExit) is raised within it. Nothing catches this exception, it is the natural indicator that the microthread is exiting. Anyway, it unwinds the call stack and the microthread exits and is garbage collected. What I would have done (and have done in other projects) is to store a reference to the action performing microthread when it is created. Then when an entity starts a new action, or gets an event indicating it has been killed, or removed from the game environment, that tasklet can be killed.
Is it possible for you to raise an exception through your generator from your scheduler, and to kill one of your microthreads in this way? I do not know.
@Ben A very good point about invariants, especially related to AI. A change that requires the actor to change state should probably cancel their co-routines in some way.
@Richard An awesome reply. I agree, using C#’s generator system as a continuation / co-routine system is a hack on top of a hack in some ways, but only Mono supports actual continuations / co-routines.
I’m thinking, in general, this system just needs a more robust scheduler that can make the connection between a running coroutine and the connected parent object, something I don’t think C# can do by default.