We’ve just wrapped up our first game that employs full Test Driven Development (TDD) practices. I’ll share my experiences, good and bad, now that we’re completely finished the first version of the project. I’ve spoken previously about Test Driven Game Development (TDGD) but a lot of that was theoretical so today I’d like to give some more concrete thoughts on how TDGD helped with the creation of Battle Group.

Test Driven Game Development?

The idea behind Test Driven Game Development is writing automated Unit Tests to confirm the correctness of your production code. Unit tests can be written to test all areas of a game from gameplay to rendering engines and anything in between. As I stated previously, there are three main goals from TDGD:

1. Find out when code breaks. If you make a code change that breaks something that was previously working (and is tested) you will know about it immediately.
2. Forces modular code.  For code to be “unit testable” it must be modular with clear boundaries and “separation of concerns” between various systems.
3. Allows you to “find the fun”.  Once you’re guaranteed that certain requirements are met, you’re free to experiment with gameplay to make it more enjoyable without breaking the game.  This also extends to giving designers more freedom when scripting.  Encouraging experimentation by designers without programmer intervention is invaluable.

How We Did It

In general the development of new gameplay features followed these steps:

1. Design the unit test for the new feature, testing the outward interface seen by the rest of the system
2. Implement the feature as quickly as possible, cutting corners as needed so the feature is playable in the quickest time possible
3. The team tests the feature, makes sure it’s working and fun and we make iterative improvements to it
4. Once happy with the feature I then refactor the code to clean up any shortcuts and make it the optimal solution

This system worked extremely well and lead to a high development velocity both for the creation of new features and maintenance/update of existing features.

Gameplay Testing

All of the unit tests created for Battle Group were testing gameplay code as this is the most complex area of our game. Unity does the heavy lifting for most of the technology based systems (eg rendering, input) freeing us to have ~90% of the code in the project related directly to gameplay. The main motivation for this testing was to allow us to rapidly develop features and experiment with existing code without breaking existing systems. For the most part, this motivation was met throughout the lifetime of the project. While the gameplay stayed fairly constant throughout the 4 month development cycle, there were a couple of major changes we made and the unit tests were invaluable during these changes.

Battle Group started out (as most games do) as a game design document. We then prototyped the game, tweaked the design document and began working on the alpha build. This design document was translated into unit tests. While I (the programmer) was responsible for this, I plan in the future for our game designer to begin to take over the role of writing gameplay specific unit tests. As the project progresses, the artifacts of its design move from a static design document/wiki to an active, maintained set of unit tests. As design changes requests came in from the team I would focus my time on updating the unit tests to match the new design.

What About Prototyping?

We did not create these unit tests during the prototyping stage (about a week’s work) as this was a throwaway prototype and unit tests would have slowed us down without much real value. One of the negatives related to TDGD is the reduction in velocity while making fairly major changes to the codebase. This is often the case while prototyping and therefore I strongly urge against TDGD during the prototyping phase of your development, particularly throw away prototypes.

The prototyping phase is a great opportunity to start fleshing out the design of your testing suite however. As you work on the core features of the game you can see where the bulk of the coding effort is likely to go and also which areas are most susceptible to change throughout the life of the project. A good example of this was the blast radius and velocity of the weapons used in Battle Group. Small changes to this had a major impact on the feel, flow and accessibility of the game. For this reason I made sure that this was both easy to change and robust in the changes I made. As velocities increased the distance traveled per frame became quite large. Coupled with this was the low physics frame rate on older mobile devices and it was crucial that we had repeatable behavior at varying frame rates and data values. As I had already planned to implement TDGD post-prototype I was mindful of these areas of code and made a mental note to test these areas first and thoroughly.

Code Coverage

Whenever I discuss TDD with someone in or outside the game development industry, there’s often a heated discussion about code coverage. Code coverage is the percentage of production code that is “covered” by unit tests. There are purists that claim you’re not really doing true TDD without 100% code coverage and there are others who say some arbitrary percentage is enough. My stance is that the game/code itself should determine the code coverage you should aim for. Sometimes certain tests are causing more trouble than good (eg changing a few lines of code requires 10 times more lines of unit testing code to be updated). Either the tests need a rethink in the way they are implemented or it’s best just to remove them.

I didn’t “watch the clock” when it came to code coverage on Battle Group. My main focus was to get the best value from my limited resources (as the sole programmer on the team). During prototyping and throughout development I noted which areas of code were critical or breaking often and made sure to get the highest code coverage possible on them. There is certainly a point of diminishing returns when it comes to code coverage which differs between projects and between systems within a project.

Test First Development

I opted for a “Test First” development style where I would create my unit test and then implement the feature being tested. This allowed me to design exactly how I thought the code should be used rather than writing the solution to the problem. I found this was invaluable to keeping a clear separation of concerns and made sure everything was as modular as possible. By thinking about the outward interface of the functionality first, it helped me get in a mindset of creating exactly what I wanted. When solving an interesting and complex problem it’s easy to lose site of the original reason for the code’s existence. Test first development focuses your efforts where they are needed most, on defining and then implementing the functionality of a piece of code as seen by the rest of the system.

NCrunch

One tool that was released during the development of Battle Group was NCrunch. This tool will completely change the way you unit test. I won’t go into too much detail as it’s a little off topic, but I strongly recommend you grab it and experiment with how it works. The whole system can be summed up in two points:

1. Unit test code has real-time inline green/red lights to indicate whether it is currently passing. Tests are continually running in the background to keep this constantly up to date
2. Production code has a similar green/red light system showing how many unit tests covering this code pass and fail (or if there are no tests at all)