Judgement Calls
Apr 19, 09:26 AM
In yesterday’s post about Quake III, I showed that a large number of frags occur in a particular room on a certain map, and we observed that there’s a pretty good spot where you are likely to make a successful kill.
You’ll notice that I didn’t say whether or not any of that was a bad thing.
Maybe the level was intended to have a few sweet spots, and maybe most of the combat is supposed to happen in a given room. I don’t know, because I was not one of the level designers for Quake III.
However, imagine that you have these metrics in a game you’re building. Your designers, who are the domain experts on what is “good” and “bad” gameplay, can look at the spread of kills on their maps and say something like, “Wow, it looks like the level is front-loaded with a lot of combat and then it’s pretty dull for the rest of the level.” After making that observation, they then have to ask the question: does this observation fit with the design goals for the level, and for our game?
For example, in my hypothetical situation above, the designer might say, “That’s a bad thing! Our game is all about constantly being engaged in combat. There shouldn’t be a lull in the action.” Or, alternately, “That’s awesome, because the last part of the level is supposed to be a somber reflection on all the combat that just happened.” Or whatever.
It’s the standard workflow for data-driven game design:
- Developers decide what metrics they want to collect.
- That gets implemented in code, and metrics are collected.
- Developers look at the metrics, either with specific goals in mind, or simply looking for outliers or trends in the data.
- Once interesting data are found, the domain experts (level designers, combat designers, QA testers, programmers, whoever) review the data against what is believed to be the desired state of the game.
- If discrepancies are found, the game is adjusted.
- (Optional) Developers expand the scope of metrics collected to get a better handle on things.
- Repeat!
So you’ll notice that as we post more stats from Quake III, we’re going to refrain from passing judgement on whether the trends we spot are good or bad. That’s up to the game designers themselves (and is sort of a moot question for a game that’s 8 years old). We’re just the folks providing the data.
