Gameplay Metrics on Mass Effect
Aug 7, 03:30 PM
There was an article about gameplay metrics published on Gamasutra today by Phillip DeRosa, the Director of QA at BioWare, called Tracking Player Feedback To Improve Game Design.
It’s a really excellent read, although a little short for my taste. While the article contains some metrics results, the meat of it is actually about the process of implementing gameplay metrics. DeRosa makes a point that I emphasize repeatedly in all of my metrics presentations: your metrics will be useless if you don’t have a good process in place.
In particular, we both agree that you need to identify:
1) an expert who knows something about statistics and the area you’re trying to answer questions about (game design, customer service, whatever). This is the “owner” of the database, or the log files, and should also be the one creating, but not necessarily designing, the reports.
2) an internal customer for the reports that will eventually be generated from the metrics. This person will dictate to the expert the kind of report they need, although obviously the expert can come in and say that something doesn’t make sense and they can tweak the final report.
3) a developer who implements the actual technology that records the data.
In DeRosa’s case, he also talks about playtesters and playtest coordinators, which is actually something that I miss when I talk about the process. This is because I’m used to collecting metrics on an MMO during closed beta, when we already have a bunch of people playing our game. But he’s right: especially for a single player game, you will need to carefully coordinate your playtests so you don’t get useless data.
Of course, I have to agree with DeRosa when he says that you should
hire an expert, define a test process, test the product with the target audience, objectively observe and recording findings, review, improve and repeat as needed.
I’d like to think we can act as that expert for your company!
