Obviously, you don’t want to start this kind of testing until most of your core systems are in place. It doesn’t have to be pretty, but enough of it should be reasonably functional that someone can play for at least 10 minutes without intervention from the observers to move them through the game.

Virgin status after a single exposure like this can be considered reset after 2-4 months, especially if new major systems or content has been added that you can have the “almost virgin” concentrate upon. They have already done the NDA thing and are more comfortable in front of the camera or with people watching them play. Subsequent exposure requires longer resets.

Designers are the most obvious group that should observe, either live or watching the tapes, but you may have to physically restrain and gag them with duct tape to prevent them leading the player or explaining the game. QA is the other most crucial group and closer to release, Marketing should get involved to hone the launch plan. You can refine the demographic and get unexpected hooks so you are more likely to get the game into the heads of your consumers for those 2-3 minutes for them to decide whether they want to purchase it. By the time your game is done, everyone is too close to the game to assess how this should be done.

Finally, and I cannot stress this enough so pre-apologies for the caps: ***VIRGIN PLAY TESTING DOES NOT SUBSTITUTE FOR ACTUAL QA. IN ANY WAY.*** They will find bugs, probably MAJOR bugs that your QA department has completely missed (and some designer will go “How the hell did QA miss that??”) but they aren’t latent genius QA…they just haven’t been taught how to play the game the way your experienced testers have. So they navigate menus awkwardly or miss as step of a quest and voila! a showstopper bug appears.

Doing this kind of testing will improve your software in unexpected and very deep ways…from the most basic concept to the interface design to game balance to art to marketing – you will get improvements across the board if everyone has an open mind to the kind of changes suggested from seeing someone struggle with your game.

I’m a little wary of handing my piece of expensive consumer electronics to a stranger I’ve met in the street, but the random feed back would be very helpful.

Maybe that’s something we could see happen at the Mana Bar opening in Brisbane? While the demographic will most likely be gamer-centric, it could be a good opportunity to get the opinion of a wide variety of gamers all in the one place.