Test Driven Game Development?

The idea behind Test Driven Game Development is writing automated Unit Tests to confirm the correctness of your production code. Unit tests can be written to test all areas of a game from gameplay to rendering engines and anything in between. As I stated previously, there are three main goals from TDGD:

1. Find out when code breaks. If you make a code change that breaks something that was previously working (and is tested) you will know about it immediately.
2. Forces modular code.  For code to be “unit testable” it must be modular with clear boundaries and “separation of concerns” between various systems.
3. Allows you to “find the fun”.  Once you’re guaranteed that certain requirements are met, you’re free to experiment with gameplay to make it more enjoyable without breaking the game.  This also extends to giving designers more freedom when scripting.  Encouraging experimentation by designers without programmer intervention is invaluable.

How We Did It

In general the development of new gameplay features followed these steps:

1. Design the unit test for the new feature, testing the outward interface seen by the rest of the system
2. Implement the feature as quickly as possible, cutting corners as needed so the feature is playable in the quickest time possible
3. The team tests the feature, makes sure it’s working and fun and we make iterative improvements to it
4. Once happy with the feature I then refactor the code to clean up any shortcuts and make it the optimal solution

This system worked extremely well and lead to a high development velocity both for the creation of new features and maintenance/update of existing features.

Gameplay Testing

All of the unit tests created for Battle Group were testing gameplay code as this is the most complex area of our game. Unity does the heavy lifting for most of the technology based systems (eg rendering, input) freeing us to have ~90% of the code in the project related directly to gameplay. The main motivation for this testing was to allow us to rapidly develop features and experiment with existing code without breaking existing systems. For the most part, this motivation was met throughout the lifetime of the project. While the gameplay stayed fairly constant throughout the 4 month development cycle, there were a couple of major changes we made and the unit tests were invaluable during these changes.

Battle Group started out (as most games do) as a game design document. We then prototyped the game, tweaked the design document and began working on the alpha build. This design document was translated into unit tests. While I (the programmer) was responsible for this, I plan in the future for our game designer to begin to take over the role of writing gameplay specific unit tests. As the project progresses, the artifacts of its design move from a static design document/wiki to an active, maintained set of unit tests. As design changes requests came in from the team I would focus my time on updating the unit tests to match the new design.

What About Prototyping?

We did not create these unit tests during the prototyping stage (about a week’s work) as this was a throwaway prototype and unit tests would have slowed us down without much real value. One of the negatives related to TDGD is the reduction in velocity while making fairly major changes to the codebase. This is often the case while prototyping and therefore I strongly urge against TDGD during the prototyping phase of your development, particularly throw away prototypes.

The prototyping phase is a great opportunity to start fleshing out the design of your testing suite however. As you work on the core features of the game you can see where the bulk of the coding effort is likely to go and also which areas are most susceptible to change throughout the life of the project. A good example of this was the blast radius and velocity of the weapons used in Battle Group. Small changes to this had a major impact on the feel, flow and accessibility of the game. For this reason I made sure that this was both easy to change and robust in the changes I made. As velocities increased the distance traveled per frame became quite large. Coupled with this was the low physics frame rate on older mobile devices and it was crucial that we had repeatable behavior at varying frame rates and data values. As I had already planned to implement TDGD post-prototype I was mindful of these areas of code and made a mental note to test these areas first and thoroughly.

Code Coverage

Whenever I discuss TDD with someone in or outside the game development industry, there’s often a heated discussion about code coverage. Code coverage is the percentage of production code that is “covered” by unit tests. There are purists that claim you’re not really doing true TDD without 100% code coverage and there are others who say some arbitrary percentage is enough. My stance is that the game/code itself should determine the code coverage you should aim for. Sometimes certain tests are causing more trouble than good (eg changing a few lines of code requires 10 times more lines of unit testing code to be updated). Either the tests need a rethink in the way they are implemented or it’s best just to remove them.

I didn’t “watch the clock” when it came to code coverage on Battle Group. My main focus was to get the best value from my limited resources (as the sole programmer on the team). During prototyping and throughout development I noted which areas of code were critical or breaking often and made sure to get the highest code coverage possible on them. There is certainly a point of diminishing returns when it comes to code coverage which differs between projects and between systems within a project.

Test First Development

I opted for a “Test First” development style where I would create my unit test and then implement the feature being tested. This allowed me to design exactly how I thought the code should be used rather than writing the solution to the problem. I found this was invaluable to keeping a clear separation of concerns and made sure everything was as modular as possible. By thinking about the outward interface of the functionality first, it helped me get in a mindset of creating exactly what I wanted. When solving an interesting and complex problem it’s easy to lose site of the original reason for the code’s existence. Test first development focuses your efforts where they are needed most, on defining and then implementing the functionality of a piece of code as seen by the rest of the system.

NCrunch

One tool that was released during the development of Battle Group was NCrunch. This tool will completely change the way you unit test. I won’t go into too much detail as it’s a little off topic, but I strongly recommend you grab it and experiment with how it works. The whole system can be summed up in two points:

1. Unit test code has real-time inline green/red lights to indicate whether it is currently passing. Tests are continually running in the background to keep this constantly up to date
2. Production code has a similar green/red light system showing how many unit tests covering this code pass and fail (or if there are no tests at all)

Introduction

I first used virtual reality in 1995 while visiting SegaWorld in London.  They had a ride you sat in, donned a virtual reality headset and furiously pressed buttons to win or lose a battle (along with 20 or so other people).  I was so excited to finally get to use virtual reality that I took a wrong turn getting on the ride and fell off into the hydraulics.  I’m not sure if it was the amazing 3D, head tracking or loss of blood from my injuries but I had an awakening experience.  I knew that in the future, we’d all be playing games using virtual reality, and the world would be a better place.  Unfortunately, this never happened, but is it too late?

Should it be the “Next Big Thing”?

VR has a number of key features that make it the obvious path for game development in the future:

• Full 3D Immersion – rather than looking at the game world sitting on a monitor that’s sitting in your room you are actually in the world.  Even 2D games take up your entire peripheral vision making them immersive.
• True 3D (for free) – Having two images allows a true stereoscopic display of the 3D world
• Head tracking – Most VR headsets have built in head tracking which adds to the immersion.  Now you can look at the pre-pubescent kid in counter strike before he headshots you.
• Look cool – Why wear nerdy 3D glasses when you can look like this:

It’s best suited to the following games:

• Flight Sims
• First-Person Shooters
• Racing Games
• Sport Games

VR also opens up the possibility of relaxation and full emersion games that aren’t possible with regular monitors.  This might sound a little hippie like, but I can see some great opportunities in games for children in hospitals and rehabilitation from injuries.

In my eyes, these benefits make it the ideal candidate for game development in the future.  The current bout of depression leading from the immersion experienced in the “Avatar” movie is proof that even 3D movies are enough to have a greater impact on people (and that some people are fairly weak).

3D movies bring up an interesting stepping stone towards VR.  Monitors are now available with high enough refresh rates that 3D glasses can be used to present a 3D image on the screen at 50Hz.  This might be an easier sell now that 3D movies are becoming more mainstream.  I find it interesting that some people are happy to accept 3D movies as the “next big thing” while dismissing virtual reality as a thing of the past, despite the fact it has much greater qualities.

So why hasn’t Virtual Reality taken off?

There are a few main reasons why Virtual Reality is still not in the mainstream of gaming:

• Price – With the average headset coming in at $1,500USD, this is a lot for an 800×600 display by anyone’s standards.  While it is comparable to some larger end monitors it’s still too high for the average gamer.  If the price of headsets can come down below $750USD and resolutions can be at least 1024×768 I expect a much larger uptake.  That’s certainly the sweet spot for price and resolution for me.
• Vertigo/Eye Strain – Some people have issues focussing on the screens in the virtual reality headset and others suffer from eye strain.  Most people should get used to this over time however it might simply mean some people never adopt the technology
• Lack of interest – People just got bored with the idea.  For so long we kept hearing about how VR was going to change the world and it never did.  A great example of this is the fact that searches for “Virtual Reality” are down 80% since 2004!

Any other crazy thoughts?

What would a blog post from me be without a reference to Microsoft?  The ultimate way for virtual reality to take off is for Microsoft to develop the headsets.  They make awesome peripherals, have the money to back it and have the console to go along with it.  Having both Xbox and PC support virtual reality would be better for gamers, game developers and give Microsoft more of an edge over the competition.

What do you think?

Have you used Virtual Reality before?  Did you like it?  How much would you be willing to pay and what resolution do you think should be the minimum supported?  Many people I’ve spoken to about Virtual Reality think it’s dead, what do you think?

Roles

Game development involves a number of common roles as well as some overlooked ones.  Individual team members can handle multiple roles on smaller teams and some roles can be shared by more than one person.

Programming

Programming on an independent team is about more than just game development.  The programming department will be responsible for tools and website development.  They will also have to handle the creation of the installer.

Design

Design is one of the great areas it pays to share ownership.  The designer has the final say on decisions and is the go to if people are unsure.  They also maintain the game design document to unify the team.  A wiki is a great way to collaborate on the design document.

Art

The art department is responsible for not just game art but also creation of marketing material and web design.  Artists need to work closely with the programmers to make sure technical requirements are met and to drive the requirements for tools.  They also need to collaborate with design to make sure the look and feel of the game is met.

Sound

Often overlooked, sound is vitally important to give atmosphere to the game.  Sound should not be left to the last minute as it will feel disconnected and won’t add as much to the game.  The programmers should build the infrastructure for the sound early to allow experimentation.

Project Management

Project managers are responsible for the velocity of the team (how much they are achieving each milestone).  They need to be aware of when team members are falling behind and make sure everyone is performing their duties.  Visibility is the key to making sure everyone on the team is aware of their current goals and deadlines.

QA

Without QA the game will be unplayable.  QA is another area that is good to share across the whole team.  The team should be playing the game constantly and fixing bugs as they go.  It is preferable to have non-programmers doing QA as it is easy for them to overlook issues in their own code.  Sharing the games with friends and having a beta-testing period is a great way to share the load.

Business

Without someone managing the business side of game development, you may as well be making the game just for fun.  One team member needs to be responsible for making sure the game will be profitable.  The entire team needs to be mindful of the fact they are making a game for profit, not just fun.  Business includes marketing, sales, business plan writing and possibly seeking funding.  The team needs a business plan covering:

• Marketing Strategy – How will people find out about your game?  How will you sell the game and how much will it cost?
• Cash flow Summary – What are the expenses of the team and how will you afford to keep the business running?
• Economic Assessment – How much do you expect to make from the game, based on similar games

Co-Location

If possible, the entire team should work together in the one building.  This lets the team quickly test ideas, keep each other motivated, play-test the game together and watch each other play.  Renting an office is expensive and it’s important to keep costs as low as possible.  Get creative, work together on laptops in a coffee shop, move in together, or work in each other garages or basements.

Workloads

Depending on the situation each team member may have differing hours they can devote to the project.  Each team member’s level of commitment should be discussed and decided up front.  The closer these levels can be the better.  Having a large imbalance in working hours can lead to resentment and hostility between team members.  Some ways of evening out imbalances include:

• Larger share/profit ownership
• More creative control over project

Personalities

Personality trumps ability on indie teams.  As with most high-intelligence professions, game developers often have strong personalities.  While these personalities might be annoying at a large game studio, it can be the death of a small indie team.  Each developer needs to put the game and the team ahead of their own agenda.  When possible a team should be formed with people that have worked together previously.

Conflict

Every team has conflict.  Conflict can be minimized having a unified goal.  The best option is to have someone in a position of power to settle disputes.  This person must be trusted by everyone not to push their own agenda.  Some solutions to solving conflict in your team:

• Talk it through – Open, continuous conversation is the best way to solve problems
• Compromise – The larger the team, the harder it is to keep everyone happy.  Compromise is a necessity.  Ultimately the team is in it together and need to compromise to solve disputes.
• Give everyone a voice – Voting is a great way to solve disputes.  The team must agree that a voting mechanism will be used and respected up front.

Outsourcing vs In-house

Many Indie companies have little money, necessitating all work is done in-house.  Outsourcing gives a reliable quality for a fixed price.  Some areas worth outsourcing include:

• Game Engines – shared by the whole team, they can save a lot of development time
• Sound Libraries – can be used as building blocks to create sounds for the game
• Art – Art is often the critical path in development.  Outsourcing may save time and money

Conclusion

This should set you up on the path to creating a solid team for your next indie game project.  What are your experiences with building a team?  Where have you succeeded and failed? Do you have any recommendations for others who are building their team?

Features

Steam Points could be used to buy games, Downloadable Content and even in-game assets (through micro-transactions).  They open up the option of gift cards rather than buying a specific game for someone.  This would be perfect leading up to Christmas as non-game savvy friends and family could buy gift cards rather than guessing and thinking “Diaper Dash” is something you’d enjoy.

All payment would still go through PayPal keeping the security burden on them.  Keeping payment with PayPal also stops Valve from watering down their focus by becoming a merchant facility.  It will simply give gamers more options with how they buy their games.

Advantages for Valve

• Create new revenue sources – gift cards and micro-transactions
• Encourage micro-transactions within games, increasing revenue
• Bringing more developers and gamers to their platform
• Another differentiator to their competitors (not that they really need one)

Advantages for Developers

• A new revenue source
• Revenue during the life-time of the game rather than only at the point of sale
• Reduce barriers for customers as the initial cost of the game can be lower
• Micro-transactions allow more styles of games to be developed

Advantages for Gamers

• Using a trust company for purchasing points.  Valve isn’t going bust any time soon
• Initial cost of games will be lower, increasing the number of games a player can afford
• More options for how players want to play (pay for convenience)
• Can purchase points at a retailer removing the need for a credit card
• Unifies the currency leading to less confusion for the international market

Advantages for Retailers

• Gives them a simple avenue into online distribution
• Can sell cards in their shops and make a cut of online sales
• Game retailers may resist however it’s similar to “iTunes credit” which most music shops now sell

Conclusion

Whether you love or hate Steam, it’s here to stay.  I see a lot of great opportunity for Valve in the future and Steam Points are a great way to begin their move forward.  Some people I’ve discussed this with are against the idea as they already see Valve as having a monopoly over online distribution.  What do you think?  Would you be happy to start spending Steam Points to buy your games and make micro-transaction?  Should other platforms like Impulse use this system to differentiate themselves?

Facts

• IWNET – matchmaking for PC users just like consoles and L4D
• Helps you play against people your own rank
• Will replace in-game browser – no other option
• No dedicated server or server list
• Rely entirely on IWNet for matchmaking and finding games
• Still has private match – customize game and invite people to server
• Can’t put it up on dedicated servers you rent
• IWNET currently focussed on matchmaking only – big plans for its future
• Games running off users PC – along with consumer grade internet
• Everything will go through IWNET
• Matchmaking NOT through steam
• IWNET runs in conjunction with steam
• No more PunkBuster, using VAC for anti-cheating
• IW want to improve Multiplayer game on PC and make it easier for everyone
• Clan matches will use private games
• Complete control over who can come in, including kicking and banning
• Changes how IW is able to update and support the PC version
• Allows more control and structure
• Good from development perspective
• Can change all features of game without having to find server or run a server
• Helps reduce piracy, but this was not a driving force for creating IWNET
• All made in-house, working on it for a while
• Will listen to feedback and make changes in the future

Verdict

This is great for casual gamers and not so great for hardcore gamers.  The biggest sticking point for most people is the lack of a dedicated server.  There are two major issues from this:

1. Less Customization. Clan matches, and many players, love their customizations.  Without dedicated servers, the extent to which gamers and server admins can mod/tweak the game is greatly reduced.
2. Lower Performance. Dedicated servers have a lot of CPU power, and more importantly, high throughput and low latency internet connections.  Consumer grade internet connections (particularly here in Australia) can’t compete with the connections of most dedicated servers.  This will result in a lower user experience and limits on player counts.

The biggest boon for casual gamers is the ease in finding a game and playing against players your their  rank.  This is great for “first-timers”.  It’s easier for them to join a game and they should be playing against other low ranked gamers.

This is the core reason I think IW have invested so much time and money into IWNET.  Multiplayer games are by their very nature viral.  If a gamer purchases MW2 and sits through the singleplayer game, there’s a chance they will tell their friends what they are playing.  However if that same gamer is converted to a mulitplayer gamer, they will actively encourage their friends to buy the game and join them.  The more gamers IW can convert to multiplayer, the larger their sales.

Solution

The podcast was fairly light on technical details, however I saw a solution that would keep the 160K+ petition signers happy, without ruining the player experience for first-timers.  Rob mentioned a number of times that clan matches, and “custom matches” can be set up through private games.  The simple solution is to allow these private matches to point themselves at a dedicated server.  This would likely take some further engineering time, however it would be a fairly small investment compared to the mammoth task of implementing IWNET.  This would allow full customization on the server side and would be an “opt-in” situation.  This keeps the user experience clean, while allowing full customization for hardcore players.

Conclusion

This is the second large petition in the games industry this year.  Have you signed either the IWNET or L4D2 petition?  What are your thoughts on IWNET and where do you see PC multiplayer gaming moving in the future?

So, Which Platform!?

The short answer is, it depends.  There’s no silver bullet when it comes to platforms.  Each has its strengths and weaknesses and the choice depends on the game you are creating.  I have a set of questions below, move the sliders to represent your situation and see the recommendation for platforms.  The further to the right the platform, the better it fits with your situation.  If there’s a platform I’m missing you’d like added, please add a comment.

The Platforms

Now that you have your recommended platforms, I’ll give some more info on my thoughts on developer for each platform.

iPhone

The iPhone market is saturated with games and is getting worse every month.  It seems nearly every indie game developer I know is making an iPhone game.  My biggest recommendation is to make something small that stands out.  The sales are very top heavy which means you’ll either make nothing or massive sales.  The biggest issue is that the only real way to get in front of people is to be in the top 50 applications which is extremely difficult due to the market saturation.

The other gotcha is the requirement for both a Mac and learning Objective-C.  There are a number of great engines out there (eg Unity) which I’d recommend using to speed up development.

Console

Unless you have funding, full version console games are almost impossible for an Indie to develop for.  It’s really hard to get your game accepted by a console manufacturer (Microsoft, Sony, etc) and the certification process can be a nightmare.

PC

PC development is the most open and an easy to break into.  There are many distribution methods (Steam, Impulse, self-publishing) to suit your games needs.  Piracy is one of the biggest threats to your game and building a game that takes piracy into account is a good step forward.  Something as simple as requiring users to login to a central server can help reduce piracy.

Web-Based

Web based games are the best place to get started, particularly if you just want to get a game “under your belt”.  It’s a great way of reaching a wide audience quickly, however don’t expect too much in either money or respect.  Web-based games are one of the hardest ways to make money from games (at this stage) and tends to have one of the most immature communities of all platforms.  Lost Garden has a great set of articles that discusses generating real money for web-based (primarily flash) games.

XBox Live Arcade

XBLA games are like a cut-down version of a console release.  It’s much easier to get accepted and get through certification.  There are also less rewards/potential for your game compared to a full version console game.  As the name suggests, XBLA is perfect for smaller “arcade” style games.

Multiple Platforms

One recommendation I would make is to start small.  Look at developing a smaller version (or prototype) of your game on one of the low-end platforms (flash or iPhone) to gauge the response to your game.  If it’s successful, then put the extra time and money into developing a larger version of your game on a higher-end platform.  John Carmack has stated his belief in developing for low-end platforms first.  This helps you focus on the core fun of a game, rather than flashy graphics or other “next-gen” features that don’t necessarily make the game better.

Conclusion

You should now have a better idea of what platform to develop your next game for.  If there’s a platform I haven’t discussed that you ‘d like to hear about, leave a comment and I’ll add it to the list.  The important thing to remember is that no platform is perfect.  They each have their own strengths and weaknesses and it’s critical that you chose the platform that best fits with your game.

Problem

When you chose an engine or platform to develop for, you’re locking yourself into its scripting language.  This has the following core problems:

• Difficult cross platform development.  Cross platform development is hard enough without the worry of porting gameplay code.  Gameplay code should not be platform specific, and therefore there is no reason to have a different language per platform/engine.
• Immature languages.  With each company re-inventing the wheel when they implement a new language designers are left with immature languages.
• Too many languages.  Designers must learn a new language whenever they develop for a new engine/platform.  This increases development time and reduces fluency in a language.  A designer cannot be free to fully express themselves until they are fluent in the language they describe their gameplay in.

Solution

The solution to these problems is to have a unified scripting language shared by all engines/platforms.  Written specifically for game development it would take the best of current scripting languages.  The games industry as a whole would “own” the language allowing it to evolve with the needs of the industry as each new generation of hardware is released.

Advantages

There are a number of key advantages for moving towards a unified scripting language.

• Single gameplay codebase.  Gameplay doesn’t change dramatically between similar platforms (e.g. 360 and PS3 or flash and iPhone).  A unified language would allow a single codebase to kept across multiple platforms helping with porting and maintenance.
• Specific to game development needs.  A language created specifically for game development could cater to the needs of the industry.
• Single language for designers to use. Small nuances in existing languages can create major issues for non-programmer designers.  A single unified language reduces these problems and makes the learning path for students clearer.
• Less Wheel Re-Invention.  An open-source unified scripting language would allow anyone to submit changes to the language to improve it, rather than re-inventing a whole new language.
• Unified Toolset.  A unified set of tools could sit on top of a unified scripting language.

Genre Specific

The core language will hold the basic building blocks for all games.  In the future, small libraries can be built by studios or the community for each specific genre.  For example a library of functions common to most FPS or RTS games could be created and shared between developers.   A unified scripting language makes this sharing possible and encourages people to develop well thought out, robust libraries they can reuse from engine to engine.  It also serves to soften the blow for designers when changing engines as their common set of functions will move across with them.

How

There are two main options for the creation of a new unified scripting language:

• Large studio forks their current language.  For example Epic could open up Unreal Script and give out the source code for its implementation (let’s call it OpenUnrealScript).
• Community Driven.  A simple codeplex/google code project with a couple of leaders and a team of developers would allow for rapid development, while meeting the needs of multiple teams.  Early adopters would drive the initial feature-set and help to drive its development.

Conclusion

My dream is to see the next generation of consoles and other platforms (PC, iPhone, Flash) all supporting a unified language.  I’m interested in people’s thoughts on this idea and candidates for which language to use.  It’s less about language features and just agreeing on a core language everyone can use to develop games and bring all developers onto the same page.