Why analytics matter – 7/22/14

Why analytics matter

By Benji Smith, sophomore, Northeastern University

As game developers, we often tell ourselves stories when making games. Not the ones that you see in a script, but ones about the people who will play the game. “Okay, so this part will introduce the player to mazing, and then they use that to defeat the boss, and then they’ll feel awesome.” Every pixel, line of code, or variable changed all has a purpose, and we tell ourselves that the player will interact with it in a particular way and get a specific reaction out of it.

In reality, things aren’t that easy. Our stories about our games are often as fictional as the stories within them. That’s why we playtest. But playtesting isn’t an exact science; it’s often much more qualitative. And that’s good! There are a lot of changes that require qualitative feedback more than raw data. But there are also a lot of risks that come with it. We tend to gloss over feedback more easily. Every piece of info we receive gets contorted into our story. It’s all too easy to assume that the tester is an anomaly, that most players will play the game ‘properly.’

But with quantitative feedback collected from all players, the illusions disappear. It’s no longer, “I didn’t see the button,” but rather “Out of 100 players, 63 didn’t see the button.” Likewise, it’s no longer “This player just doesn’t understand,” instead “63 percent of players just don’t understand.” This summer, we’re using Splyt Analytics to help drive our decision making on Midnight Terrors.  It’s hard to argue with the data, because data doesn’t tell a story. Data is the story.

That’s not to say data is infallible. It can be outdated or incomplete, and it can certainly be misinterpreted. But it can never be wrong. That’s because data is just what happened. It doesn’t carry any analysis of its own, it’s just a series of numbers and events.

In some ways, it’s a bit intimidating to use analytics.  It’s effectively handing off your game, and letting whatever will happen, happen. You can’t coach people, and you can’t hide from the results. In some ways, it’s like launching a title (just with a quarter the stress). But it’s absolutely necessary, because polishing a game can’t just be an art. It needs to be an art and a science. And for that, we need data.