Category Archives: Programming

DirectX Programming

Finding D3D9 Leaks

April 12, 2012 – 10:45 am

I ran into this problem working in a DirectX 9 application recently, and found it useful enough that I thought I'd share. We were running into an issue where returning from an alt-tab was not recreating the device properly. This is usually caused by a created D3DPOOL_DEFAULT asset not being released and recreated before attempting to recover the lost device. As a result, you get this message:

Direct3D9: (ERROR) :All user created D3DPOOL_DEFAULT surfaces must be freed before ResetEx can succeed. ResetEx Fails.
Direct3D9: (ERROR) :ResetEx failed and ResetEx/TestCooperativeLevel/Release are the only legal APIs to be called subsequently

One additional thing that was throwing me was this error message:

Direct3D9: :Window has been subclassed; cannot restore!

This can probably safely be ignored, and may be a symptom of the back buffer leaking specifically, but generally this whole error set is a sign of leaking D3D objects.

If this is something innocuous, like a single texture, you can sometimes figure it out by simply enabling the DirectX debug libraries, running and exiting your application, which will give you the AllocId of the leaking asset, which you can then break on. However, a large number of leaks indicate that something lower level may be leaking, and potentially a leaking reference to something, like the back buffer (which was our case). How to find it?

Here's what I ended up having to do.

First we did what we need to do for every bug, reduce it to the fewest steps possible.

Next, run PIX and take a full capture, until you've gotten to the point of the leak, then close. If you have a lot of steps to reproduce, you'll want to make sure you've got a lot of hard drive space, and some time on your hands, because it can take a while.

Once you have your full PIX run you can look at this column:

And set it to Never. This will show you every D3D object that's leaking, regardless of how many AllocIds it has (interesting tidbit, a single D3D object can allocate multiple AllocIds). If you move to the last frame (and wait for PIX to catch up, which, again, can take a while) you can even see how many references exist during shutdown. You could even walk each frame and see when the reference counts of your leaking assets go up.

In our case, like I said, we were leaking a reference to the back buffer. Most obvious was to look for calls to IDirect3DDevice9::GetBackBuffer, but none of those were leaking. Less obvious was to look for cases of IDirect3DDevice9::GetDepthStencilSurface and IDirect3DDevice9::GetRenterTarget(0). In our case, a call to GetRenderTarget was leaking as a result of a short circuit in a logic check, resulting in the back buffer reference to dangle in a very specific logic case.

Granted, this leaking does not pose a lost device recovery problem with DirectX 10 or DirectX 11, but since many engines are still DX9 based, I thought I would share.

Permalink | Comment (0)
.NET Games / Design Mono Open Source Programming

Debugging Embedded Mono

March 30, 2012 – 11:16 am

It's been announced that Go Home Dinosaurs will be released using Google's Native Client technology later this year, but what isn't known (as much) is we're using Mono / C# as our "higher level" logic and scripting system. Generally, I'm a huge fan of C# as a language, and the performance benefits just seal the deal for me. The one issue we have right now, though, is debugging embedded Mono, especially in conjunction with the game running in Visual Studio. There aren't very many instructions on how to do this online, so I thought I'd share our experience.

There are two very weird issues when debugging embedded Mono. First, it currently works exactly opposite of how most debuggers work. In most soft debuggers, the application starts the server and the debugger connects to it. In Mono soft debugging, the debugger creates the server and the client connects to it. Second, the Mono soft debugger uses signals / exceptions to work halt threads. This means that you can't debug both with Visual Studio and Mono Develop, because Visual Studio will pull the signals before they get to the application. This also prevents

However, there are changes in the works from the Unity team which fix both of these issues. However, they are not currently part of a stable mono release (or in the Elijah's version for Native Client). Once they are, I may revisit this whole article, but that's where we are right now.

So, with all of that in mind, how do you get this working right now?

Set Up MonoDevelop

First up, grab MonoDevelop.

Second up, you will want to add an environment variable to your system called MONODEVELOP_SDB_TEST and set it to 1. This gives you access to the following menu item when you have a solution loaded:

Which brings up this dialog:

Now, for some reason, I've never been able to get MonoDevelop to launch the application directly. It always fails attempting to redirect standard out, and then immediately crashes (I could look at the code to figure out why this is, but I haven't had a chance to). We'll come back to this dialog in a minute.

Prepare Your Code

There are a few things you need to add to your code to get debugging to work. The first few lines can occur at initialization, regardless of whether you're debugging or not:

mono_debug_init(MONO_DEBUG_FORMAT_MONO);
mono_debug_domain_create(mMonoDomain);

These should be done before you load any assemblies.

In addition, if you're running Mono from a background thread (like we are), and not running it on your foreground thread, after initialization you need to detach the foreground thread. This is because when Mono attempts to halt all threads when it hits a breakpoint, it can only do so when you're executing Mono code. If your main thread never executes any Mono code, it never halts, which means debugging doesn't work. This is good practice regardless, because if the GC needs to halt all of your threads, it will also fail if a non-Mono executing thread is still attached.

Last thing you'll need to do in code is set up the debugger. Here's the code:

     bool isDebugging = AresApplication::Instance()->IsDebuggingMono();
     const char* options[] = {
          "--debugger-agent=transport=dt_socket,address=127.0.0.1:10000"
     };
     mono_jit_parse_options(1, (char**)options);

This needs to be combined with any other options you may have (I only have one other, which is –soft-breakpoints), and should only be done if you want to start debugging. If the Mono client can't connect to the debugging server when it starts up, it will call exit, which is no good.

Lastly, if you're using the Microsoft C# compiler, instead of the Mono one, you'll need to generate mdbs from all of your pdbs. There's thankfully a utility for this. We actually now perform this step as a post build event, which keeps the mdbs up to date with latest code.

Put it all together

Alright, so let's get this party started!

Open MonoDevelop. Set some breakpoints, then open this dialog again.

Don't change anything, and click "Listen".

Now start your application outside of Visual Studio with whatever command line arguments you chose to signal to your app you want to debug in Mono.

Everything should now connect and you should be debugging Mono in your app!

Let me know if you have any issues!

Permalink | Comments (5)
AngelXNA Games / Design Programming XNA

GGJ 2012: Eat Sheep (and Die)

January 30, 2012 – 12:41 pm

This weekend I participated in Global Game Jam for the third time (after taking a hiatus in 2011 to go speak at conference overseas). This year, I was at the UVA site and took part in a 3 person team with my brother in law and an awesome UVA student. The result was an interesting two player competitive game called Eat Sheep (and Die).

This jam was way less stressful than any other jam I've participated in. We essentially had something fun to play by middle of day 2, and just polished things throughout the end of day 2 and day 3. As a result, this is probably the most complete jam game I've ever worked on. Very little went wrong. It was great.

So what when right?

  1. AngelXNA – Power of Angel combined with the power of XNA! This was the third time I've used Angel in a jam and every time it's improved. We were able to get levels in games quickly, spawn actors, and do all sorts of things quickly and easily. It actually worked really well. Again.
  2. Abandoning Design – The original point of the game was to have a cyclic back and forth of helping and hurting the other player. On your turn, you would have your goal, and a directive to help or hurt your opponent. On the next turn, you would be given a bonus based on how well or poorly your opponent did. At some point, we realized there was never any reason to help your opponent. It just wasn't worth it. So we abandoned the helping, turn based component, and went with real time versus. This makes it questionable whether we obeyed the theme, but whatever. The game was much better as a result.

What went wrong?

  1. AngelXNA? – So, the one thing I don't like about Angel and XNA is that it does have a huge barrier to entry for people playing the game after the Jam. I know I don't bother playing a GGJ game that requires a download unless it comes VERY highly recommended. I will, however, look at pretty much every HTML5 or Flash game. That said, Angel is awesome, so maybe I have to figure out a way to fix this. Native Client maybe?

That's it. Not too many lessons this time out because everything went so well. It's a weird feeling. Really looking forward to next year!

Permalink | Comment (1)
.NET AngelXNA Programming XNA

Updates to AngelXNA

January 22, 2012 – 10:57 pm

One of the tips I gave in my recent talk is to have a tool for GGJ. My tool of choice is AngelXNA, because it combines the power of XNA with the power of Angel. When I was demoing it to the other attendees, I realized I hadn't added a few demo screens that are really necessary.

To that end, I've added two demo screens showcasing two features that are in AngelXNA that aren't in Angel. The first is the Editor. While I had a problem with the Editor in the last GGJ, Fire Hose used it a lot for prototyping and I think we've worked out most of the kinks. It's not perfect, but it does make creating levels in Angel super easy. There are shortcuts for cloning, scaling, and rotating actors, which makes the level editing process super quick.

The second feature has several aspects, but comes down to the ability to use sprite sheets and the ability to create named, data driven animations. There are two parts to this. The first part is the Tiling system. By adding a .tileinfo file parallel to any given image file, it will automatically start rendering Actors that use that image as tiled. Here's an example .tileinfo:

IsAnim = true
FrameSize = Vector2(160, 200)
AnimDirection = TiledAnimDirection.Horizontal
StartFrame = Vector2(0, 0)
EndFrame = Vector2(0, 2)

It specifies that this file contains an animation, that each frame is 160x200, that the animation frames run left to right, starts at 0,0, and ends at 0,2. You can also use a non-anim and specify a specific tile to render.

Once you have these .tileinfos specified, you can specify animation files. Here's the example anim file:

BeginAnimation("Idle", TiledAnimRenderPath.Create())
    AnimDelay = 0.03
    File = "images/devo/devo_idle_strip"
    AnimType = AnimationType.Loop
EndAnimation()

BeginAnimation("Run", TiledAnimRenderPath.Create())
    AnimDelay = 0.03
    File = "images/devo/devo_run_strip"
    AnimType = AnimationType.Loop
EndAnimation()

BeginAnimation("Fall", TiledAnimRenderPath.Create())
    AnimDelay = 0.03
    File = "images/devo/devo_fall_strip"
    AnimType = AnimationType.OneShot
EndAnimation()

This allows you to call just Actor.PlayAnimation("Idle") or Actor.PlayAnimation("Run").

This is still a bit more complicated than I'd like, but does work really well. The only thing I'd like to improve is the specification of the render path. AngelXNA actually generally selects the correct render path automatically, but the data driving system, specifically the animation system, isn't detecting its render path. I can probably fix this, which I'll hopefully do after GGJ.

Permalink | Comment (0)
Games / Design Programming

On Singletons

September 14, 2011 – 11:07 am

Inspired by a Twitter discussion with Liam Devine about my previous blog post, I've decided to take a bit of time to talk about my opinion on Singletons. In my previous post, I used a lot of them, though generally, I'm not a huge fan. In fact, I've been known to dedicate non-trivial amounts of time in talks to the subject of why singletons suck. Singletons are just globals under another name, and they have almost all of the same drawbacks as globals, and don't provide too many benefits. Each singleton is a potential bottleneck, a potential threading nightmare, and almost always creates giant side effects throughout the code base every time they're used. In addition, if you're doing unit testing, use of singletons / globals requires significant set-up overhead, which can make tests more complicated.

So what gives? Why am I using them here?

Generally, there's nothing intrinsically wrong with a Singletons or globals. It's their overuse that becomes a problem. As mentioned above, each singleton needs to be managed carefully because it can become a potential bottleneck across your entire code. However, when you're working with something that does need to be global, a Singleton is sometimes a better option, since you can insist certain conditions be met before an instance can be returned.

A good example is a Fixed String Table. If you're unfamiliar with this, here's the idea: every string in your code is translated to a hashed number. In some cases, this translation is done manually, but for the sake of code readability and simplicity, I do the translation in code any time a FixedString object is created. In order to make sure that same string always returns the same hash, and to make sure we can translate the hash to a string, we put the string into a table. This table could be a static member of FixedString, but if you want to, say, tell the table where to allocate its memory from, or how much memory it's allowed to use, you need to initialize said table. Again, you could use a static method for this, but this is hiding what's actually happening. Furthermore, if your table class is any amount of complex, or has functions for accessing usage data, you're essentially going to be pushing that all into static methods of FixedString. It's a pain, when you are, in essence, creating a global interface anyway, since FixedString itself will probably be in global scope. At least by moving things into a singleton you potentially have the option of creating multiple tables in the future, should you so choose.

But what about all those Factory singletons from my initial post?

First, a point of clarification: in the previous post, I made it look like every factory was a singleton. I did this for simplicity, and I apologize. It actually isn't true. My factories aren't singletons. In reality, the code reads like this:

Application::Instance()->GetFactory()->RegisterComponent(…);

Application is one of the other few singletons I have in my engine. Now, I'll admit that this isn't much better than having the factories themselves be singletons, since they're retrievable from a singleton anyway. If you look at the code, it looks like my component code is now coupled to both the factory AND the application.

But in reality that's not true. My factories aren't coded or tested as singletons. If I wanted to create another factory I could, and all my unit tests will still work. And the components themselves aren't coupled to the factories or the application at all. The only thing that couples factories, the components, and the application is the auto-registration system, which serves as a bridge between all three systems. Each system is still usable stand alone.

This just leaves whether or not the Application as a singleton / manager is a good idea. The answer is, no. No it isn't. But since it is one of few, it sticks out like a sore thumb when you type it, and in code review. Every time you see or type Application::Instance() should be like warning lights going off in your brain that you're potentially doing something dangerous. BUT, since I'm mostly doing this during initialization, and at no other point, I'm okay with it.

Permalink | Comment (0)
Programming

Getting Around Indeterminate Initialization

August 24, 2011 – 10:38 am

So, as I'm sure many seasoned veterans of know, the initialization order of static variables is undefined by the C++ standard. While this generally isn't a problem, it can quickly become a problem if you're doing something like overriding the memory manager, as is the case in most games. Any structure that wants to call out to a custom allocator, must wait for the allocator to be initialized. One way around this is to have the allocator as a self-instantiating singleton, but what if you're doing something more complicated?

An example is a self-registration system. This is a fairly common occurrence: you have a factory class that takes care of instantiating other classes based on something data-driven. In my scenario, this is a component system, but it really could be anything. Each component needs to register itself with the factory class so the factory can proceed to instantiate them. You have two options here: one is to have a giant init method that looks something like this:

void InitFactory()
{
     Factory.RegisterComponent(ComponentA);
     Factory.RegisterComponent(ComponentB);
     Factory.RegisterComponent(ComponentC);
     Factory.RegisterComponent(ComponentD);
     Factory.RegisterComponent(ComponentE);
     // and so on...
}

The problem with this is that it requires you #include every component header, and remember to register every component manually. Not my idea of fun.

The second is to use an auto registration system. Basically, you create an auto-registration template class:

template <class Component>
class ComponentAutoRegister
{
public:
	ComponentAutoRegister();
};

template <class Component>
ComponentAutoRegister<Component>::ComponentAutoRegister()
{
     Factory::Instance()->RegisterComponent(...);
}

And then add that to any component class:

class MyComponent
{
    static ComponentAutoRegister<MyComponent> __AutoRegistration;
};

When the static auto-registration template class is constructed, it will automatically register your class with the factory.

The problem is this: If your factory class has any other dependencies, it will need to initialize those first. If this usually happens post-main, or as part of any other variable being statically initialized, you can't use this auto-registration system. Damn.

Here's my scenario: I have a table of fixed strings that needs to be initialized with a table page size and an allocator before it can be used. My component factory uses fixed strings as keys for each component. Instead of attempting to make a long chain of singletons which would only end up coupling all the systems together (something I wanted to avoid) I slightly edited my auto-registration system to auto-register a callback instead, which is then called post initialization:

template <class Component>
class ComponentAutoRegister
{
public:
	ComponentAutoRegister();

	static void PerformAutoRegister();
};

template <class Component>
void ComponentAutoRegister<Component>::PerformAutoRegister()
{
	Factory::Instance()->RegisterComponent(...);
}

template <class Component>
ComponentAutoRegister<Component>::ComponentAutoRegister()
{
	Application::AddPostInitHook(&PerformAutoRegister);
}

The callbacks are held in an opaque linked list, managed by a zero-overhead pool pulled out of system memory. It could actually be held in an array as well, but being able to free the memory once post-initialization is complete makes me happy.

And this could easily be extended to support multiple layers of auto-registered post-initialization, so long as the layers are kept free of dependencies. For now, I only need the one layer, so I'll keep it simple.

This may seem hacky, but it does work pretty well. What are your thoughts?

Permalink | Comments (2)
Activism Breaking In Industry Programming Tools

Upcoming Talks

January 16, 2011 – 6:44 pm

Just to keep people in the know, I'm giving two talks in the coming months. The first will be at Games Forum Germany (site is in German, obviously, thought the talk will be in English) on either January 27th or 28th, I don't really know which. The talk is titled "Data Driven is Half the Battle" and will be talking about why just setting up data driven code isn't enough, and how (both from a high level and a low level) you can create a system for giving others on your team the ability to edit your data quickly and easily.

The second talk will be at GDC, as part of the Game Career Seminar, which takes place on the Friday of the conference and is focused at students. That talk is called "From College to Industry: 20 Lessons for Getting the Most out of your Early Career" and is about, well, getting the most out of your early career. It is basically a lessons learned talk about my 10 years attending GDC, and my 6 years in the game industry. While that isn't a really long time in the game industry (anymore), I feel that some of the lessons I've learned over that time will only really be apropos to students for a short time, and thus it is most useful for me to give the talk now, when it will be most useful to them. I will be practicing this talk twice in front of student groups around the Boston area, once at WPI and once at a local SIGGRAPH meeting, to make sure that the talks really hit all the right notes and lessons and are as useful as possible to the GDC student population.

Both talk's slides will be posted on line after the talks, and the GFG talk will probably be cross posted to the Toolsmiths, since it is a tools related talk.

Permalink | Comment (0)
.NET AngelXNA Games / Design Programming XNA

Simplicity in AngelXNA

November 7, 2010 – 11:16 am

I know I've promised tutorials on AngelXNA, and I actually have the first one written, but I've been distracted by crunch on my current project and don't feel comfortable just posting the first portion without more to follow. In addition, the first tutorial feels way too long, so I'm hoping to add some minor changes to Angel to make getting started even easier, specifically by setting up GameManager and default screens in ClientGame, rather than forcing you to make them yourself.

But enough about that, this post is about rendering.

Angel is really nice in that it makes a lot of things really simple and really transparent. With a few exceptions, everything does what you expect. There are very few pre-conditions for any given call in Angel, so you don't have to worry about whether you've set up X or Y before calling things, or worry that calls will fail because objects weren't in the right state. That just doesn't exist In Angel. With that mind, I'm looking for a way to add a piece of complexity and functionality to Angel without sacrificing the external simplicity.

In AngelXNA, all actors render as a rectangle by default (we didn't port over circles from Angel C++), so by just putting an actor in the scene, you get something rendering. If you want to make it a sprite, you call Actor.SetSprite or (simpler) Actor.Sprite = "ContentFile". Animations follow a similar pattern, loading a sequence of files using Actor.LoadSpriteFrames (or, again, you can use Actor.Sprite = "ContentFile_001" and it will automatically load the sequence). Then you can play the animation sequence using Actor.PlaySpriteAnimation.

The problem is, now, I want to add two additional render paths, one for rendering static and animated sprite sheets, and one that allows you to specify and load multiple named animations (so you could say Acor.PlaySpriteanimation("Jump")). This is easily done by either inheriting from Renderable or Actor, but inheriting from Renderable loses all of the functionality in Actor, and inheriting from Actor means I can't use these render paths in other sub classes of Actor (namely PhysicsActor). In addition, if these render paths were reusable *outside* of actor, it might make alleviate some problems we've had adding things as Actors that aren't just to get their rendering properties.

So my question is, how do I make this simple? One option is to just add more methods and properties. Actor.SetSpriteSheet(), Actor.SpriteSheet, Actor.LoadSpriteSheetAnim() etc. This does have the benefit of being consistent, but doesn't make those render paths reusable. I could create an IRenderPath interface and have one for each render path, using SetRenderPath to assign a specific render path to an Actor and remap the current functions to creating those render paths. In addition, it means the SpriteSheetRenderPath could make use of a reusable SpriteSheet for both tilled maps as well as character animations. This, however, I feel makes Angel overly complicated. Maybe some combination of the two makes more sense?

I feel like I've spent way too much time thinking about this, and would love to get it implemented, hopefully soon.

Permalink | Comment (1)
AngelXNA Games / Design Indie Games Programming Tools XNA

Immigration Jam Post Mortem

August 23, 2010 – 6:41 pm

This past weekend was the Boston Immigration Game Jam, and like previous Jams, I'm going to do a quick post mortem about my experiences, not only because I think it will help others, but because it will help me as I prepare for my next game jam. For this Jam I was in a team of 5 on day one, which included 3 musicians, which later went down to 4 when two of the musicians decided to not show up for day 2 and one new programmer joined the team.

The game we ended up making was called "Cultural Exchange" and was supposed to be about managing a country, trying to attract the "right" sort of people while keeping out undesirables. To do this, you would enact legislation to make your country more appealing to some people, and erect fences and the like to keep out undesirables. However, people in the country wouldn't like things being too restrictive, since they value their freedom, so it was a constant balancing act.

Now I know normally I start with what went right at the jam, but this time I feel what went wrong is more important. So here we go:

What Went Wrong

  1. Not having a clear idea: While the description is fairly clear, neither me nor Kevin had any idea how we were going to get it working. We didn't actually get anything designed until very late in the night, and even that was just kind of border-line. Since we didn't have a clear idea about how the game would play, we had no idea what our restrictions were.
  2. Using Flixel: Now, no offense to Flixel, but I was not impressed with my first outing with it. This is partially linked to not having a clear idea of what we wanted to do, and not having a clear understanding of the limitations of Flixel. I spent 90% of the first day fighting with ActionScript and Flixel, not only because I didn't understand Flixel's limitations, but because Flixel has some issues with transparency, some of which it inherits from Flash. I spent at least 3 hours trying to move some buttons off screen, only to throw up my hands in frustration. In addition, we were using FlashDevelop, which does not have a debugger included for Flash, which made dealing with all the problems that much worse.

    I took 3 learning points out of my Flixel experience, which don't actually relate to Flixel at all:

    1. Always use something you're familiar with for a game jam, even if it's not the best tool for the job.
    2. Constrain your idea to what you know is possible in the tool you're working with.
    3. Do not take platform into account. If you really like your game after the game jam, you can always port it to a different platform.
  3. Too big a scope: Out game had way to big a scope for our time frame. There was just no way it was getting done.

What Went Right

  1. Switching to AngelXNA: At about 10 on Saturday, I switched all of our code from ActionScript to C#, and all of our Flixel code to AngelXNA. I was too pissed with Flixel's limitations to deal with learning how to get around them, whereas I knew the limitations of Angel really really well, and I could get around them very quickly (see point 3a). Even though this meant losing a programmer on the project, it still ended up well (see the next point).If nothing else, Angel, I've realized, is EXTREMELY transparent. I find pretty much everything does what you'd expect. That said, getting started up in Angel is hard, and I do need to do some tutorials for it in the near future.
  2. Having a dedicated designer: The second day, Kevin ended up doing only design. This was the ONLY way we actually were able to finish at all. Kevin ran through turns on paper, and looked into how legislations would affect the population on each turn. Looking back at other Jams, I think we could have done better if we'd had someone we dedicated as "designer" to look into these problems earlier, rather than playing it by ear the whole time.

Generally this was a short Jam, and I went into it already tired, so I wasn't super psyched about it. But I think I learned a little bit more about jamming, and some things I really want to add to Angel for the next jam, so keep an eye out for that.

Permalink | Comments (2)
AngelXNA Experimental Games Games / Design Programming XNA

One More GGJ Observation

February 3, 2010 – 11:57 am

Today I realized another thing that went wrong with our GGJ product, that I felt needed sharing.

  1. Starting At the End: This one is twofold. First, we spent a lot of time working on the end boss battle of the game, which, while awesome, had to be integrated with the rest of the game at the end of the jam, mostly in the last hour. The core mechanic of the game, the destruction and rebuilding of objects, suffered as a result. Second, when the game was pitched, I included a portion "at the end" where it turns out you're a horrible person for destroying the world. We never got to this because the game took precedence.

Basically, I've found that almost any game jam game that has the words "and at the end of the game" never gets to that point because developing the core mechanic and building gets in the way. And if you need the end of the game to keep with the theme of the jam, you're going to end up breaking theme.

I will say in defense of my team that it was mostly me talking about the "end of game" theme. Amanda pretty much spent the entire time telling me we wouldn't get to it, so this rest squarely on my shoulders.

In the end, I don't think Quest For Stick fit the main theme of the Jam (deception), but I think the core mechanic was awesome, and I'm happy we decided to make it instead of making something that would obviously fit with theme.

Tags | Permalink | Comment (0)

Next Page »

Next Page »