BioShock Needed Better Metrics!
Nov 14, 07:33 AM
The latest issue of Game Developer Magazine features a post mortem on BioShock written by project lead Alyssa Finley. I’ll quote this point from the “What Went Wrong” section of the article verbatim:
Poor data collection. One of the most frustrating things about our decision to be more data-driven in tuning the game was the lack of actual good data to base that tuning on. Our game log system was barely adequate to analyze single playthroughs and became completely unwieldy when trying to analyze a single log file containing data from multiple playthroughs. We had no good methodology to define what information was logged and at what level of detail, so the job of parsing out the logs into understandable “gameplay metrics” was painful, slow, and ended with inadequate results.
As single-player games become more data-driven, you need to have better data collection. I couldn’t have said it better myself.
The important part is the methodology part. This is why our Aleph Metrics Suite is driven by an XML file where you strictly define all the data you’re recording. It’s a really great system: it forces you to sit down and really think about what you’re going to record and why.
And people: log files suck. They’re totally inadequate. If you’re dumping your gameplay metrics to a log file… you’re almost better off not wasting your time dumping gameplay metrics. At best they’ll be annoying to use, and at worst they’ll actually obfuscate what’s really going on.
