Orbus Gameworks

Academic Article on Game Analytics and Achievements

Nov 2, 07:15 PM

Here’s a highly academic but interesting article on Generations of Game Analytics, Achievements and High Scores. It takes a broader view of analytics than I normally would, which is kind of interesting. Don’t expect to get anything practical out of it, but it’s a fun read if you’re a closet egghead like me.

— Darius Kazemi

Halo 3 Visualization Critique

Nov 8, 08:18 AM

The Visual Methods blog has some interesting analysis of Bungie’s consumer-facing Halo 3 visualizations. I wonder how these visualizations compare to in-house visualizations of private data sets?

— Darius Kazemi

Comment

Social Network Metrics and Idea Infection

May 3, 04:21 PM

Let’s say you want to hold an in-game event in your virtual world. In fact, let’s say you’re doing a sci-fi MMO, and you want people to know that the DCJ MegaCorp is giving away experimental weapons to anyone who shows up to their corporate headquarters and says the password “salamander.”

In my mind, there are three ways to go about doing this.

1. Send an email to your user base, telling them where and when the event will take place.
2. Post to your forums with the same information.
3. Spread the idea through word of mouth.

Most people know how to do #1 or #2, but… how do you organize #3? It’s actually not that difficult, provided you’ve been collecting player metrics on how your social networks are organized.

Social Network Metrics

To collect good metrics on your game’s social networks, you really don’t need much more than the following table:

Player1	Player2
Alice	Bob
Alice	John
Alice	Mary
Bob	Alice
Bob	Lisa
...	...

That’s it. It’s just people’s friends lists. If Alice has Bob on her buddy list, create a row for them. Tada. Now, you can always augment this if your game has types of relationships or something, but this is almost all you need, at least for the purposes of viral marketing.

Given a table like the one above, you can do a simple SQL query on it to figure out who has the most friends. Now take the top 50 people out of that list, and tell them about your event. But make it cool. Since you’re only contacting 50 people, you can probably put a customer service or OCR rep on the task, and have them cook up a neat rabbit-hole style invitation. Disguise the admin as something cool in your game, a big mech assassin or whatever, and then hand them a little script and give them some room to improvise. Make it fun! But most importantly, make it so that you really leave a big impression on the 50 people you contact.

Once you’ve done that, 50% of your game world will know about the event in no time!

Don’t believe me, huh?

Some Simulation Models

I wrote up a simulation in Prefuse, the most excellent graph visualization toolkit in the world (it’s free, too!). The first thing it does is generate a random social network. My algorithm is along these lines:

1. Given N nodes (people), divide those into social groups of 1 to 40 people (uniform distribution).
2. Give each person within those social groups on average 7 close friends within the group (0 to 14, uniform distribution)
3. Give each person on average 2 random acquaintances

That’s it. It’s not a perfect network generation algorithm by any means, but what it does is it models the fact that human networks consist of cliques (in MMOs they typically max out at 40 people or so in a given guild) where everyone has a high chance of knowing everyone else. And then I throw in some randomness because in virtual life, as in real life, we often pick up random friends for no good reason.

I was happy to see that the graphs it generated actually looked more or less like the real social network graphs I used to visualize when I was working on large-scale MMOs. Here’s a small graph (200 nodes) which visualizes nicely. (Once you get to about 500 nodes or more, visualizations turn into spaghetti.)

The Simulation

Now what better to do with a social network graph than to run simulations on it? I wrote a quick-and-dirty sim which takes 4 parameters.

1. Threshold is the minimum number of friends you need to have in order to be considered a viable target of our viral marketing event.
2. Spread1, Spread2, and Spread3 are probabilities. Spread1 tells you how likely the person you contact will tell any given one of her friends. Spread 2 is the probability that those secondary people will tell their friends. And Spread3 is the probability that those friends will tell their friends.

So Threshold defines how many players we are contacting, and the SpreadX parameters define how “strong” our viral marketing is. I decided to limit it to 3 layers of friends, to keep things simple.

In our simulation, we assume that there’s an 80% the person your admin contacts will tell their friends, there’s a 40% chance that their friends will do so, and a 20% that the people who heard it third-hand will do so.

The Findings

In a network where people have on average 7 close friends and 2 random friends, we found that you don’t need to talk to very many people before word gets around.

In this table, the first column is the number of people in the sim, the second is the number of people that are contacted directly by an admin, and the third is the percent of the total population that gets exposed to your event.

Population	# Contacted	% Exposure
1000	7	30
2000	6	20
3000	10	20
4000	8	12
4000	21	26
4000	100	61
5000	9	12
5000	24	26
5000	59	44
5000	140	66

These numbers might not look so great. But consider this. My friends in marketing tell me that if you send an email to all of your paid subscribers you can expect about a 5% return on that email. So that’s 100% exposure (via spam), and a 5% return. But I would wager that while you might only be getting exposure to 10% or 30% of your population through this seeded word of mouth, those people who hear about it will be

• more likely to attend your event
• more likely to ask around about future cool events
• more likely to attend future events if they end up being the first one contacted instead of just hearing about it

In the case of getting 30% exposure, I still think you’d get about a 5% attendance rate to your event, out of the cool factor alone.
Now, I personally think these exposure numbers are low, that these ideas will spread better and faster than our simulation indicated, for the following reasons:

• We limited ourselves to three layers of word of mouth. In practice, it’s unlimited, although with diminishing returns (usually).
• The 80/40/20 ratio might be better or worse, depending on how cool your seed event is. We’d like to think you can do better.
• We don’t account for the simple fact that as the worlds get bigger (more nodes), people will have more random acquaintances. We kept it at a firm 2 acquaintances all the way through, when really we should have been increasing the number along with the total node count.

In fact, if we go into the sim and take our 3000 node network and increase the average number of acquaintances from 2 to 4, while seeding the same number of people (10), we jump from 20% exposure to 35% exposure! Just goes to show: in graph theory, edges mean everything.

So What?

If you’re an OCR (Online Community Rep), you already know all this. A well-run in-game event that targets the right people will spread like wildfire around the game world. Nothing new here.

Except that metrics can help you pick who you contact initially. You can use metrics to identify the pressure points of your social network, letting your in-game event get the maximum impact for the minimum effort.

— DariusKazemi

The Bots of Quake III

Apr 30, 11:34 AM

Hi, this is Craig. I did the recent visualization stuff, and I do most of the analysis. Particularly, I’ve done a lot of analysis on the Quake 3 bots and how they perform.

The common stuff was easy: the bots are definitely separated into good, mediocre, and bad for each difficulty tier. Which bots are good and bad differs from difficulty to difficulty because each tier for each bot is hand-coded. For example, Klesk dominates at low difficulty levels, but loses to Crash (a notoriously easy bot) on nightmare. This is simply because Klesk isn’t scripted to dominate at the high levels.

Some bots are better at specific map types, too. Xaero, the end boss, is a kill machine on open, underpopulated levels. But he’s thoroughly mediocre (at least at moderate difficulties) in enclosed, crowded levels. Why?

Well, if you’ve played Quake 3, you’re probably thinking “Xaero likes railguns. Railguns are only really good at long range. So of course in a crowded map he’ll be mediocre…”

It’s true that Xaero theoretically wants the railgun twice as much as the BFG. But that’s not the reason.

Surprisingly, we found that bots follow the same weapon preferences. The reasons for this are kind of foggy, but it appears that the bots only switch weapons when their current weapon isn’t optimal… and the rocket launcher is considered optimal in every situation. Which means that all the bots tend to use basically two weapons: the rocket launcher and their starting machine gun. Their personal weapon preferences (such as Xaero’s addiction to the railgun) cause that weapon to come in a distant third.

What this means is that Xaero is not better at open levels because of his weapon preference. His weapon preference hardly matters at all, at least in bot-dominated games.

Each bot has a huge number of stats, but when it comes down to it only a few of them really matter: reaction time, aim skill/accuracy, and attack skill being three big examples. Which of these stats is most useful depends on the situation. Reaction time is most useful when you’re likely to have a surprise visit from someone with a shotgun. Aim skill is great for those long shots. Attack skill is good for open brawls.

The rest of the stats – field of view, various tactical weights, tendency (and skill at) chatting, tendency to grab items… all of them are, at most, barely noticeable. They have no reliable effect on games of four or more players.

Unless they are suddenly, amazingly effective in team matches or duels, these stats serve no purpose other than a slightly erratic attempt to make the bots seem a bit more human.

... Wish I wish I’d known that when Quake 3 came out. I spent a lot of time tweaking those stats in a vain attempt to give my bots more combat personality.

I have a feeling that the Quake 3 scripters did the exact same thing.

— CraigPerko

Comments [3]

Judgement Calls

Apr 19, 09:26 AM

In yesterday’s post about Quake III, I showed that a large number of frags occur in a particular room on a certain map, and we observed that there’s a pretty good spot where you are likely to make a successful kill.

You’ll notice that I didn’t say whether or not any of that was a bad thing.

Maybe the level was intended to have a few sweet spots, and maybe most of the combat is supposed to happen in a given room. I don’t know, because I was not one of the level designers for Quake III.

However, imagine that you have these metrics in a game you’re building. Your designers, who are the domain experts on what is “good” and “bad” gameplay, can look at the spread of kills on their maps and say something like, “Wow, it looks like the level is front-loaded with a lot of combat and then it’s pretty dull for the rest of the level.” After making that observation, they then have to ask the question: does this observation fit with the design goals for the level, and for our game?

For example, in my hypothetical situation above, the designer might say, “That’s a bad thing! Our game is all about constantly being engaged in combat. There shouldn’t be a lull in the action.” Or, alternately, “That’s awesome, because the last part of the level is supposed to be a somber reflection on all the combat that just happened.” Or whatever.

It’s the standard workflow for data-driven game design:

1. Developers decide what metrics they want to collect.
2. That gets implemented in code, and metrics are collected.
3. Developers look at the metrics, either with specific goals in mind, or simply looking for outliers or trends in the data.
4. Once interesting data are found, the domain experts (level designers, combat designers, QA testers, programmers, whoever) review the data against what is believed to be the desired state of the game.
5. If discrepancies are found, the game is adjusted.
6. (Optional) Developers expand the scope of metrics collected to get a better handle on things.
7. Repeat!

So you’ll notice that as we post more stats from Quake III, we’re going to refrain from passing judgement on whether the trends we spot are good or bad. That’s up to the game designers themselves (and is sort of a moot question for a game that’s 8 years old). We’re just the folks providing the data.

—