Category Archives: Programming

.NET Experimental Games Games / Design Industry Programming

Jamming Postmortem

February 2, 2009 – 11:28 am

I took part in the Global Game Jam this weekend, and I have to tell you, it was a lot of fun. Version 1 of the game we created, The Game Of Nom, is available from the Global Game Jam site, and was voted third favorite at the location we were participating in, and I think that was a fair place for it to be (Move Mouse To Fulfill Destiny and The Beat were really awesome). I'm really happy with how the game turned out. It had the right feel and I think it really extracted the emotions from players that we wanted. The rules were simple enough that you could easily sit down and play it, hard enough that you could play for a while before winning, and interesting enough to be fun. That all said, the game is fairly buggy, especially when you're moving around flocks or trying to combine them, and that's a huge detriment to the game. At some point, Darren or I may actually fix a few of the issues and post a new version on the game jam site, but don't hold your breath.

So, for my own sanity and for future reference for everyone, I thought I'd do a post mortem of my experience.

What Went Right

  1. Enlisting the full time help of an artist. Amanda did an amazing job of giving us a feel for the game very early. I have no doubt that without her, the game wouldn't have been nearly as fun or interesting, and wouldn't have achieved this balance of fun and message that we wanted. By having a cute style to the game, we were able to present the dark message without seeming overly pretentious, which was awesome. My new rule is "artists make things look cool quickly," so get them involved early and things will look cool early, and get everyone really energized for the rest of the jam.
  2. Having a team. The first game jam I participated in, it was just me. Now, that was great for rapid iteration, but not for making something really interesting. I didn't have anyone to bounce ideas off of, and no one to really keep me focused and in line. Working with Darren not only allowed us to do something a little bit more complicated than we would have been able to do alone, but also ended up producing a much better product.
  3. Not sweating the small stuff. For the most part, I think we did a good job not worrying about some of our problems until later, and getting the game playable quickly so we could test it and refine it as needed, instead of spending lots of time doing things like improving the flocking behavior (which, I'll admit, I spent a little too much time on anyway ;) ). The key to Jams is knowing when things are "good enough," and I think we did a pretty good job with that.
  4. Tools choice. Although we had some problems with it, XNA/C# is a really great prototyping language. Right before the Jam, Darren and I were considering other options, including the beta of Unity that was made available to the jammers. The things was, we didn't want to spend lots of time fighting to get things on screen and working, when we could spend time on the game play. XNA didn't give us a lot of pain for our simple little 2D game, and for that we were pretty thankful.

What Went Wrong

  1. Needing a prototyping framework. XNA is awesome, but it's not a great prototyping framework. As I don't do too much prototyping, I really don't know what I need and what's overkill. I found that the two things I really ended up wanting / needing were a simple object manager and an actor framework / state machine framework. We actually implemented states very late in the process and they were very hacked together. I found myself wishing we'd had OnEnter / OnExit / ChangeState for the little blobs frequently, but implementing states would have taken more time than hacking around them. In this respect, we maybe should have gone with Angel which has that stuff already built in, but it'd come out the day of the Jam, and I didn't want to try to learn it while Jamming (I've learned my lesson from the OLPC jam).
  2. Clear message, not so clear implementation. We knew what we wanted to get across to the player early, but not how to do it, and trying to discuss it mid jam was hard. Another twenty minutes talking about implementation would have helped, though during our initial discussion I was itching to get things running. What we really should have done is a "stand up" style meeting when everyone arrived in the morning to discuss where we were, and where we wanted to be each day. I think it would have helped a lot.
  3. Not enough testing / balancing. We should have pulled in more people to play the game earlier, and should have gotten things for Amanda to play so he should see the results of her art changes quickly. As it was, I spent most of Saturday and Sunday balancing, but was so close to the game that I missed little problems. Having just one person play mid-day Saturday would have exposed lots of problems that could have been fixed by the deadline.

I'd love to know what people have to say about the game. We're rating well, and I think if we get around to fixing the bugs, it will rate even better. Thanks to everyone who organized the Game Jam for this great opportunity!

Permalink | Comments (3)
Activism Industry Tools

Introducing the Toolsmiths!

January 26, 2009 – 10:10 am

So, if you haven't noticed, this blog has been a little quiet recently.  It's not because I don't have a lot to say, but because I haven't had the time to say it.  One of those reasons for that is because I've been spending a lot of time working with the IGDA Tools SIG (which I've talked about before) to launch a new collaborative blog: The Toolsmiths!

The new blog has two other great authors, Geoff from Insomniac and Dan from Robotic Arm Software.  I'm really looking forward to the content we'll have over the next year or so, and I hope anyone interested in game tools will take some time out to read our posts.

My first post, on continous integration and build systems, will appear on Wednesday.  Hope you enjoy the new blog!

Permalink | Comments (2)
Programming User Interface

Playing it Tight and Loose

January 20, 2009 – 10:23 am

So this post has nothing to do with games, and more to do with just real world usability of applications. It's a rant I've been talking to people about for a while, but I decided to get it up on my blog for people to discuss in the greater cloud of the web. Generally, the post is about the battle between Microsoft, Apple, Google and Open Source. Not from a doctrine perspective, but entirely from a usability perspective.

The Apple Approach

So, without a doubt, Apple has some of the best usability in the market. There are lots of reasons for this, but the one I want to focus on is their tight application integration. Apple products understand other apple products really well. Syncing an iPod or an iPhone through iTunes is ridiculously easy, because it's designed to be easy for 90% of the customer base. That's why people use it. And no other company can control the hardware and the software the way Apple can.

Then there's the other 10% of the customer base. People like me that would rather use other applications to sync their music collections to their iPod or iPhone. This is where Apple fails. In an attempt to make sure that you always use their software with their hardware, they've unnecessarily (and sometimes very purposefully) obfuscated the way an iPod sync works. Certainly, certain products can do it, but from my understanding these products have reverse engineered the iPod database, and this reverse engineering frequently breaks when Apple releases new versions of their firmware.

Now, Apple has a good reason for this. They want things to be as simple as possible for the 90% of people that own or want an iPod. But what they don't do is make it easy for the other 10% that want an iPod, but have use cases outside of the bounds of what they offer.

The iPod / iTunes integration is just one example, but Apple does this all the time. Very rarely does Apple offer a product that communicates in a standard way unless that standard is so ubiquitous that they have no other choice. So Apple is tightly integrated, with no loose coupling.

The Open Source Approach

Open Source has the opposite approach. Loosely couple everything and have the programmers sort it out. This means, a lot of the time, that the average user is completely screwed. Figuring out how to get two systems to talk to each other in the Open Source world generally requires some wicked voodoo magic. Everything understands everything else, but only if you're really careful and tell them where and how to contact each other.

Now, this is a blanket statement, and I know many applications have improved in the Open Source world concerning this, but the problem is that when you allow everything to communicate freely, you lose any possibility of creating a reasonable user interface that would have come about through two products having tight integration. It's great that every open source calendar product understands iCal, but if you can't get them to communicate easily, what's the point?

The Microsoft Approach

Now, Microsoft is interesting because it's schizophrenic when it comes to integration. Individual Microsoft products are tightly integrated with a strange variety of loose coupling (or at least have been in the past). Applications like Office, Windows Media Player, Visual Studio, and others, are very tightly integrated with themselves, and they offer the options for extension and integration with other applications, so long as you're working within a very tightly confined area that Microsoft has defined.

This is basically Microsoft's history: offer a lot of developer support and allow extensions for almost everything you do, but only so long as they're developing for your platform. Only in recent years has Microsoft opened their data exchange systems so that applications outside of the controlled Microsoft environment can potentially understand what's going on.

The problem is that outside of a single application, Microsoft doesn't integrate with itself at all, and depends on third parties to offer features that are prominently displayed in Apple software. Is this bad? Well, for the average user… absolutely. Say, for example, that I want to order a set a pictures from an album I've made in my favorite piece of photo software, or let's say I want to publish them online. This can easily be accomplished from both Microsoft's offering and Apple's offering. The problem is that Apple has an offering that it owns and prominently displays as an option. Microsoft, on the other hand, would allow you to sync to any number of picture website systems, but you'd have to do some searching. The average user can't be bothered with that, and even if they could, the number of options is way too overwhelming. The average user is more likely to get confused than actually published their pictures online.

In my opinion Microsoft's approach to letting 3rd parties integrate into (almost) any piece of software they make is the "better" option, as it encourages independent development. The problem is that it only really helps power users, and hurts their general interface design. So, with Microsoft, you don't get the benefits of either the tight integration or the loose coupling.

The Google Approach

Just when you though Microsoft was schizophrenic, along comes Google. Google boasts over 13 products that integrate with your Google account, and they… well sort of, kind of, not really integrate together. Mail allows me to look at my documents and calendar (through a Labs extension), as well as allows me to add information to my calendar by detecting information in emails. But the whole system feels very disjointed. More importantly, the communication protocol between the applications is proprietary and hidden (so far as I know).

Things become more interesting when you talk about the new G1. Here's a phone that's based on an open standard for communication between applications and wants them to behave like they're tightly integrated without using tight integration. But apparently the first thing the G1 does is force you to sign in with a Google account, something you can't change. What if I want that phone, but not a Google account? Really, what was the point there?

The problem with Google is that it offers a lot to users in weird chunks. There's no unifying vision like there is with Apple, and its developer support (at least for its applications) is spotty. So you don't get external extensions and you don't get tight integration. It's the interface of any given application that draws you in, but is anyone really happy with how it works beyond that?

The End All Problem

This all comes to a head in one way: the management of your multiple personalities. My name is Jeff, and I have 3 identities on the internet: Jeff as employee and co-founder of Orbus Gameworks, Jeff as an IGDA volunteer, Jeff as just me. In some places, I want to have access to all of these personalities at once (say on my phone). In some cases, I want to share just select information with other people. I want to share my business calendar with my business associates, for example, and I want to share my personal calendar with my friends. I want my phone to have access to all my business contacts and personal contacts, and sync them to the proper places, and I want my music to stream be able to sync to my home computer and maybe my work computer without issue.

The problem with tightly integrated software is that it either can't or doesn't understand this concept. You are you regardless of what you say. The problem with loosely integrated software is that you don't get the usability of power of integration. You have to play the game tight and loose. Tightly integrate across platforms and across communication lines while communicating via loose protocols that anyone can understand.

Will this ever happen? Who knows. The problem is that each company has to fight significant hurdles to make a tightly integrated, loosely communicative application suite. Apple has to open up communications and fight against their urge to make integrated suites. Microsoft has to integrate more, communicate more loosely, and become less schizophrenic when it comes to application design. Google has to decide what they're doing, and Open Source has to get some user interface designers (seriously guys, you're killing me). It's an uphill battle for everyone.

Permalink | Comments (2)
Breaking In Industry Programming

Programmer Portfolio Tips

December 3, 2008 – 8:52 pm

Okay, I'll admit, I'm sometimes a little hard on articles on Game Career Guide. Which is funny because I've had two articles posted there
myself. So, I thought it only fair that I point out that this article on creating a game programming portfolio is actually very good. Also, since I think most of my readership is programmers, and interestingly a good number of student programmers, I thought I'd point you to it.

I do have two very minor criticisms of the article though. First, I think a lot of really good demos can take the place of one complicated game.  Certainly I always want to see a complete game, but some of the examples he puts down are actually fairly complicated.  Asteroids or Breakout, so long as it's complete, is fine.  I'd rather see something original, but it's not necessary.  If you have a lack of good demos, though, you should go for something more complicated in your full game.

Second, I don't think he puts enough emphasis on how important providing good, clean source code is.  It doesn't have to be perfect, but even if your Breakout game works, if it's complete spaghetti code it won't earn you any points in my book.  Make sure the code you provide shows a good mind for organization and good software engineering principles. That's what I'm more interested in anyway.

Any professional game programmers have additional insight for my student readers?

Permalink | Comment (1)
Programming Version Control

A Binary Problem

October 24, 2008 – 9:58 am

So, as evidenced by some earlier posts, I'm completely enamored with distributed version control, especially Mercurial. This past week, I actually was able to use Mercurial the way it's supposed to be used. We had to do a quick fix for a client while on-site, and now that we're back, I've pulled the fix onto my local box, where I can actually write the unit and integration tests to make sure we actually have the problem fixed.

However, this trip has also confirmed something that I kind of knew was a problem with Mercurial and distributed version control in general: it doesn't handle assets well at all. Mercurial even gives you an error when you attempt to add files larger than 10MB.

"Files over 10MB may cause memory and performance problems"

Though this isn't really a problem for most projects (though there are libs that can certainly get up to 10MB in size), it's a serious problem for games, where version controlling assets is a significant issue. In addition, because of the way Mercurial handles files there are limits on file size. If anything, this could be a deal breaker that would (and should honestly) prevent distributed version control from being adopted widely in the game development community.

But is it a solvable problem?

In some ways, yes it is. The simple solution is to use another piece of version control software for your assets. This is actually more common than you might think, as certain pieces of version control software offer things like asset diffs, something a generalized piece of version control software usually can't do. That said, having your team (at least your programmers and producers) learn two different pieces of version control software can be a pain, and configuring them to both play nice can be even more of a pain. In addition, if you're doing something similar to the Kernel Practice (having subsystem maintainers, leads, or "lieutenants" in charge of reviewing systems before pushing them to an "authoritative" repository) or something similar to Controlled practice (which is a hierarchical development model), having a second piece of version control software that doesn't support these does you absolutely no good.

So the best solution is to use Mercurial for your assets still, but how?

Unfortunately, there's still no good answer for this. Certain extensions (like Forest, Trimming History, Shallow Clone, and Overlay Repository) look promising for avoiding performance problems on a developer box, but not for avoiding the problem on the server. However, those in combination with Rebasing, could allow you to keep only portions of history in a given repository, but this would have to occur behind the scenes as much as possible.

The problem is that Mercurial is designed for lots of small files, so the generalized extensions are just specializations for lots of small files. So, the solution, in really, is to create an extension for Mercurial specifically designed for assets and large files, that understands that full histories are not always an option (or a necessity), that cloning full asset histories are expensive, and that a standard diff algorithm can be hugely expensive, and thus should be done through incremental diffing. And its handling of all of this should be largely transparent to users, especially since, let's face it, most artists can't be forced to deal with this stuff.

Will I make this extension? Unfortunately, no, not any time soon. As a programmer at a middleware company, we don't have many assets to version, so Mercurial works fine for our needs. That said, if I ever get back into game programming, you know step one will be to make sure I can use Mercurial for assets.

Permalink | Comments (2)
Industry Programming Tools

Tools of the Trade

October 2, 2008 – 9:40 am

I gave a talk at Becker last night on tools in the game industry, which I called "Tools of the Trade." The talk and its slides are available here, and the entire talk should be in the slide notes. Generally, I'm pretty happy with the talk itself, though if I had to give it again there are a few things I would change, including removing some of the LOLCats. In addition, I think some of the points that I make about the problems with debugging tools actually applies to all tools, but in an attempt to split the talk into three nice sections, I ended up muddling things up a bit.

What I really hope is that the students actually got something from the talk. I'm afraid some of it may have actually gone over their heads, and portions of the talk may need simplification or better explanation. Hopefully they did get something out of it (and if any Becker students have decided to read my blog, I'd love to hear your thoughts!)

Permalink | Comment (1)
Industry Programming

Slighted, yet again

September 29, 2008 – 2:49 pm

I got confirmation today that my proposed talk(s) for GDC have been passed on this year. In this case, I was hoping to get a roundtable on build process into the mix, and I find it really unfortunate that they passed on it. Generally, the "Technical Issues in Tools Development" roundtable (that John does an excellent job moderating every year) spends half of its time talking about build process best practices, which means neither build process nor technical issues in tools development actually get the time they deserve. I talked to John about this, and he believed it was a good idea to get both roundtables more focused. That said, he was pretty certain I didn't have chance of getting it accepted (and honestly, I was pretty sure he was right).

What I find most disappointing is why such a round table was rejected. These are the criteria sited by the selection committee:

  • Your expertise in the subject matter proposed. They have no way of knowing my experience with build systems, as they only ask what positions I've held at various game companies. They know I was an Associate Programmer at Bethesda and Lead Architect at Orbus. That doesn't show them that I designed (or helped design) the build systems at both companies, so in reality, this wasn't and couldn't have been a reason for rejection.
  • The number of years you've been in the industry. Going on 4 years now. I guess seems longer than it is. Really though, does this affect my ability to lead a discussion on build process?
  • Game credits and/or projects you've worked on. Sure, the build system for Aleph Metrics isn't amazing (doesn't have to deal with assets), but I ran in to any number of problems optimizing the assets for Obivion and Fallout 3 (though, to be honest, I'm not sure how much of my asset optimization code is still in Fallout 3). Again, though, there's no way for them to know what I actually did at Bethesda (they didn't ask) so really they're just looking for name recognition of projects. Would think those two would be pretty high up there.
  • Past speaking history; if previously spoken at GDC your evaluation scores are taken into account very seriously. I spoke at GDC once, sort-of. It was a student action roundtable, which kinda died before it got off the ground. That's fine. Whatever. But since then I've spoken at any number of other schools and events, and I've never had a complaint, and quite a few compliments. I mean, I'd love to talk at GDC if they'd give me a chance.
  • The relevancy of your session proposal to seasoned game developers, why is it interesting? How is it important to game development? What will game developers get out of the session? How can the selection committee think this wasn't important or interesting? Technical Issues in Tools Development is ALWAYS full and more than HALF of the time is dedicated to this and we NEVER come out with clear answers or any best practices because the time isn't actually there to talk about it. Would having an extra half hour help? I like to think so.
  • How well you addressed any of the topics outlined in the submission guidelines. This, of course, could have been the key. Maybe I'm not so good at selling myself?

Every year, I just get more and more frustrated with submission process. There's little to no feedback as to why a talk was rejected and, while I can see it with some (any time I propose a game design talk, for example), some really good submissions, like this one seem to get thrown out without actual explanation. I think the only way I'll be okay with this rejection is if another, more experienced developer, proposed the same talk.

What about everyone else? Have people heard? Are you just as frustrated as I am?

Permalink | Comment (1)
Programming

On Build Systems

September 20, 2008 – 3:15 pm

The guys over at Power of 2 Games (their blog, by the way, is probably one of the best blogs on coding process and practice that I know of) have a post that basically lambasts the SCons build system, and for good reason: it's dependency checking is slow, and for large game projects that can have very large numbers of dependencies (especially in assets) that's a problem. But, it got me thinking about build languages and build systems in general, and what I would want from a perfect build system.

As a result of the Power of 2 Games post, I took a look at not only Scons, but as many of the build systems mentioned in the comments, including Waf, Jam and it's
variants, CMake, MSBuid, Ant, and Maven. There's a ton to choose from, and most tools are happy to explain to you why they're the best (SCons expliation vs. Waf's explination). Of course all of these different build systems try to solve one problem: a complicated build process is hard.

Interestingly, I don't like any of these build systems. I use MSBuild day to day because I use Visual Studio day to day, though I use Ant for the actual build script that is executed by my continuous integration system, and for running unit tests and deployment scripts (one thing I've learned over the years is never have a target in Visual Studio that deploys an executable, it's only asking for trouble). However, the only reason I use Ant at all is because I know it, it's almost universal in its acceptance across IDEs (except Visual Studio) and because I've yet to find something that I really like more, that has a simple enough syntax and is easily extendable. I certainly have my problems with Ant, including its use of a data definition language as a scripting language, it's speed, and the fact that it can be very difficult to get it to do what you actually want, but I live with it, much like I think most people live with whatever build system they use.

So, what do I want from a build system? Why don't I like these others? I want a build system that does the following:

First, I want something that can either a) be generated from Visual Studio project files or b) is just natively understood by Visual Studio, and I lean towards 2 as not only being more possible, but the better solution. Why do I want this? Well, believe it or not, I really like Visual Studio. Despite its flaws (and it has many, especially in the C++ department), it's a pretty good IDE, and I like using it, which means I want to be able to manage my projects from *inside* it. Most build systems recognize that Visual Studio is used and thus go the other way (generate projects from build files) but can't go the other way. The problem is that MSBuild (and VCBuild) aren't very good build systems and they're extremely complicated, which means creating something that understands both MSBuild AND VCBuild is probably something that would make someone want to tear their eyes out. Having Visual Studio open and understand another format is probably just as nasty, but perhaps slightly more possible with a Visual Studio add on. The difficulty would probably be duplicating as many of the features from Visual Studio in the project file type, which could be potentially a big issue.

Second, I want a build system that understands high level project dependencies, as well as low level file dependencies. When builds rely on shared libraries and such, it's sometimes good enough to know that the shared library changed, meaning that regardless the project will need a rebuild. In some cases, a build system can look for the binaries of a changed library, instead of requesting to build source that you might not have. Maven is fairly good at this, though I don't agree with how it's done (though, in defense of Maven, it's almost certainly the way it should be done for Java).

Third, I need my build system to be cross language and cross platform. At best, it should understand how to convert build parameters between different platforms automatically, which would mean you could build using Visual Studio's system or GNU C using the same build file, same parameters. This may, however, be a complete pipe dream.

Lastly, I want a build system that can, without any changes to the build file, distribute builds among multiple machines. This is not much of an issue because it is currently true of KJam, SCons and (potentially) Waf. The idea is that the same build file could be used on a build server for complete builds of a project, and thus it may need to be farmed to other machines in order to produce a build in a timely manner.

Now, if that's too much to ask, I have other demands as well, especially on the side of continuous integration or build server. I've yet to find a continuous integration product that I really like, as very few allow one key feature that I want: Organization of a large project into multiply dependant sub-projects. If I check in a change that changes dependant libraries, I want my build system to recognize this and build each part that it needs to build. In addition, I want it to deploy that new build only when all outdated parts of the project are finished. In most cases, I think that continuous integration systems expect your build system to handle this, but…

In addition, I find that finding a system that understands build steps is also hard. For example, I have unit tests and integration tests on one of my projects. I want the unit tests run on every build, including developer builds. The integration tests, however, only need to be run by people currently working on those systems and by the build machine itself. So, why can't I tell my continuous integration system: "Build, then run these tests, then deploy. Any of those steps failed is a failed build." Instead, I have to put this in the build script… even though it's technically not part of the build.

And really lastly, I want my CI server to understand isolated builds. Team City does this (I think) and it looks like it could be pretty awesome. Let's face it, if we set up our build system right, it'll be able to build way faster than a developer machine ever could, and for a long, complicated build, a developer should be able to request that the CI take over and make a build just for him, using his code. Integrate this with Mercurial, and it could actually work really well. Couple that with artists being able to ask for asset builds, and you'd have a happy, happy team.

So, maybe this is too much to ask for a complete build system, and maybe we'll never see it, but I will say that I'd love to work on it someday.

Permalink | Comments (3)
Programming Version Control

Mercurial Second Impressions

August 21, 2008 – 9:06 am

I have a bit more to say on distributed version control, some theories on how it might be used in large agile teams, but I don't necessarily want to touch on that today. I've decided I really need to read the rest of the Mercurial book (I read most of it!), as there's things in there that help explain how the extension mechanism and hook system works, and how you can use them to promote pre-push process. Right now, though, I quickly want to respond to a few of my initial impressions of Mercurial:

  1. Local branches and rebasing are supported in Mercurial through extensions. The transplant extension, Rebase extension, and Local Branches extensions are available and allow you to do everything I mentioned. This is the benefit of using a piece of source control that has a plugin architecture and is easy to extend. You get extensions for everything. That said, I'm convinced I wouldn't use any of them (save potentially rebasing) now that I've worked with mercurial more.
  2. There's an extension for Mercurial that provides an ascii graphical log of branches / merges. It's pretty awesome. I'm sure there are other GUIs available as well, and hey… maybe I'll make a WPF one in the future?
  3. The internal branching system works fairly well, but after using it I've decided they're almost not worth using. Instead, I'm now using separate repositories, even for long lived branches. This means not necessarily knowing which branch a change comes from, but I'm okay with that.

Also, in the comments, Programmer Joe commented that he'd been thinking about distributed version control, but wants to use me as a guinea pig. Unfortunately, I'm in a very small team of people, so I don't think I'd be that usefully as a guinea pig for large teams. I certainly have theories on how a large team structure should work (which I will post about), but nowhere to try them out. I'd love to do some large team work with Mercurial to see how it'd work out. Anyone know a large team looking to be adventurous? ;)

Permalink | Comment (1)
Open Source Other Programming Version Control

On Distributed Version Control

August 14, 2008 – 10:45 am

So I had a conversation last night with my good friend Steve about my decision to start using Mercurial. Talking to him, I realized I hadn't really posted much on what I think about distributed version control systems (DVCS), so the switch to Mercurial may have taken many people off guard. So, I wanted to spend a post (maybe two) talking about what I see as the advantages of DVCS over a traditional "central server" (VCS) mentality.

The Leap of Faith

Like many, about the idea of version control without a central server scares me. I don't trust my own machine, or my developer's machines, to be in any way fault tolerant. My server, on the other hand, has a RAID 5 controller in it, and is backed up off-site weekly (though if I were checking in more I'd probably back up nightly). Having that central server keep track of my changes feels safer, so I shied away from DVCS, opting instead for centralized version control with Subversion with user branches (more on that here). Even with tools like svnmerge though, Subversion utterly fails at merge tracking, especially from multiple branches bidirectionally. It was just never scaled to do that. Since then, I've watched several videos on the design of distributed version control (two on Git (here and here), one on Mercurial) and I made the leap of faith to attempt to see what DVCS is like.

The Centralized Model, Distributed

Now, distributed version control is amazing for open source projects, but what about working in a company where team communication and process is key? Where coordination is a requirement? Well, the nice thing about distributed version control is that it can, if you need it to, support a centralized model. Even in open source projects, you have what you call an "upstream" server, which is what everyone consults for the latest and greatest "official" changes to a product. You could, if you wanted to, push every commit you made to the upstream server, and pull every so often to make sure you have the latest version. In that case it would be exactly like using centralized version control (except it takes a more hard drive space, as you're holding a repository on your hard drive, not just the source code). Sure, you have to train people to commit, then push, but once they understand that, there's no issue. In addition, your centralized version of your tree can have pre/post-commit hooks just like any other version control system, allowing you to create check-in gauntlets should the need arise. That said, I'm sure other source control providers make it easier (the new Source Safe and Perforce I believe have GUIs for create check-in gauntlets, but I could be wrong). Still, with an easily extended system (like Mercurial), these additions wouldn't be hard.

Encouraging Experimentation and Checkpoints

So in this case, why use a DVCS if it's just going to work like centralized version control. Well, even in this situation, you now have more options available to you than you did before. First is the ability to checkpoint whenever you want, without affecting other developers. The key reason most people give for using version control is the Chinese proverb "The palest ink is better than the best memory." So what not increase your memory by encouraging your developers to commit as often as possible? And if they can do this even when their build is broken or when they're not completely finished with something, why can't we allow them to do this? In a centralized model, this is almost impossible (though packing apparently solves this partially). Then, when a developer is done, their commit can have all of the changes they just made, with a history of what they did. Of course, it doesn't have to (rebasing is always an option) but having it there can help you see potential problems or flaws in thought process.

In addition, distributed version control can encourage experimentation in branches. The problem I have had with branches in the past is that they are a pain in the ass to manage on a centralized server. Even Subversion, which makes branching easy, doesn't manage merging well at all, especially bi-directional merging. I've even had problems with Perforce merging in the past. Regardless of that, though, working off of branches and creating branches is never easy. Not as easy as it is in distributed version control environments anyway. In Mercurial, branches are just clones of the current repository. You can work in them, share them with others, and delete them without the central server being involved. This may sound bad, but it does encourage developer experimentation and sharing. If I can branch simply off of whatever my current state is, experiment a bit, show that change to other developers and get feedback, then merge back into my main development branch easily, that opens up a lot of opportunities for small team experimental work.

Here's an example of where I could have used this in the past. One of my former companies did not do and did not encourage unit testing, and I could not convince my coworkers (or the management) to let me try it in some of our newer libraries. I decided, though, to do unit testing on any new modules I wrote anyway. I made a copy of the library into a directory that had a unit test environment set up, worked on the library (writing new tests, making them pass, etc), then copied my changes back over when I was done, and committed them to the central server. The unit test folder, though, was completely unversioned. If I'd had a DVCS, I could have cloned the library, done my work, then pulled changes back, leaving the unit tests versioned in their own folder. In addition, any other developer that was also working on those libraries could have easily pulled the unit tests from me, thus spreading a new practice at the company. This may be an edge case, but it is an example where simple branching and merging would have encouraged experimentation between developers.

Shelving, Packing, Branching

This gets into the second option you have with distributed version control. DVCS makes it easy for developers to share uncommitted changes with each other. Other source control providers give you the option through "shelving" or "packing", but I can't see it being as easy as it is with distributed version control. I can pull changes from anyone, on any branch, into a cloned version of my own repository to test things out without talking to a central server. Those changes don't need to be based off of the same trunk and can easily be merged by whichever developer at whatever point, then pushed to the upstream server, all while retaining who made the original changes. This has to be experimented with to really see the full benefit, and honestly I haven't done it enough, but I can see where it would be useful. Unlimitted simple branching with the ability to push and pull changes from any developer coupled with a strong revision history just sounds nice to me.

Fault Tolerance

The last thing I want to talk about is when things go wrong. We try to avoid it, but every so often, our central server goes down. Sure, we have backups off site, maybe we have a passive failover server, but if we don't, our central server going down is very problematic. I have had this happened to me, and at fairly large companies that can afford lots of nice servers. If this happens in a traditional VCS, development (basically) stops for the day. Not so much in distributed environments. Since I can commit to my local repository all I want and share with everyone else without the benefit of a central server, a dead upstream doesn't affect me at all. This, in my mind, is pretty awesome.

More…

I will say that I'm using a DVCS in a very small team environment, but looking at it, I believe it could scale very easily. I also think there are lots of interesting ways to use DVCS, especially in agile environments where small teams may need to communicate changes without affecting central efforts. I really want to go into these, but this is already too long, so maybe I'll talk about them tomorrow. Until then, let me know what you think.

Permalink | Comments (6)

« Previous Page — Next Page »

« Previous Page — Next Page »