Why Most Video Game ‘Aggression’ Studies Are Nonsense

Do violent video games make people more aggressive? Politicians and pundits have been asking that question for years now, and although everyone thinks they know the answer, scientific studies have yet to come up with results that satisfy even the most basic probing.

Last night, the gaming website Polygon reported on a studyfrom the American Psychological Association that concluded there was a link between violent games and aggression. “The research demonstrates a consistent relation between violent video game use and increases in aggressive behavior, aggressive cognitions and aggressive affect, and decreases in prosocial behavior, empathy and sensitivity to aggression,” read the report.

But a closer look at the APA’s study (pdf) leads to a number of questions—many of the same questions Kotaku asked in January of 2013, when we ran an extensive look at the current state of research on video game violence

Some context: the APA study, which was published yesterday, is not based on new research but instead is a review of studies conducted between 2005 and 2013. In addition to looking through several individual journals, researchers with the APA scrutinized four meta-analyses—reports that look at wide swaths of research and try to spot patterns—and ultimately concluded that there is indeed a pattern of test subjects growing more aggressive after they play violent video games.

There are serious problems with the study, though—problems we’ve pointed out in the past. Let’s go over a few.

1) How do you even measure aggression?

An outside observer might wonder—how can you tell whether someone is “more aggressive”? Is there really a way to measure an emotional state like aggression? Well, some of the tests used in violent video game studies include:

A) The “short story” test, where a subject is given the beginning of a writing prompt (“A driver crashes into Bob’s car. Bob gets out of his car and approaches the driver.”) and told to fill in what happens next.

B) The “noise” test, where a subject is asked to press a button that delivers a terrible sound to another subject, then evaluated based on how much noise they deliver and how intense it is.

C) The “hot sauce” test, where a subject is asked to dole out hot sauce to another subject and is evaluated based on how much sauce they give and how spicy it is.

Other tests ask subjects to fill out questionnaires asking how aggressive they feel, and if all this has you raising an eyebrow, you’re not alone. “Aggression” is an ambiguous psychological concept—if I get mad at a game and scream at my TV for a few seconds, am I being aggressive?—that can only be measured in subjective and often arbitrary ways.

2) Nobody’s looking at short-term vs. long-term effects.

One major problem with the tests used by these studies is that they all measure their subjects’ aggression directly after they’ve played violent video games. Even if you assume the tests are good ways to measure aggression, this is not particularly useful information for practical purposes. If you’re a parent who wants to know how violent video games might affect your children, the bigger concern is how their behavior will be impacted in the long run.

But there aren’t enough studies on the long-term effects of violent video games. Admits the APA in their report: “However, the meta‐analyses we reviewed included very few longitudinal studies, and none of those that were included considered enough time points to examine the developmental trajectory of violent video game use and associated outcomes.”

So the APA’s conclusion—that there’s a consistent relation between violent games and aggression—is misleading at best. What they’ve actually concluded is that there’s a consistent relation between violent games and short-term aggression.

3) Few people are thinking about one of the most important factors: competition

Many of the studies examined by the APA’s report look through a wide variety of violent video games ranging from Mortal Kombat to Grand Theft Auto to Call of Duty. Often, researchers split up students or test subjects and ask some to play violent games while others play non-violent games. But there’s one factor they often don’t consider: competition.

Back in 2013, researchers at Brock University published a longitudinal study (monitoring 1,492 adolescents over four years) that tested out the effects of violent competitive games, violent non-competitive games, non-violent competitive games, and non-violent non-competitive games. Ultimately, they found that competition was a more relevant factor than violence.

“We found that playing more hours a day of the two types of competitive games did predict aggression over time,” Brock researcher Paul Adachi told me then. “Whereas playing non-violent, non competitive games did not. So that really gets at the idea that, well, it may not be the violence, it may be the competition in games that is responsible for a link between video games and aggression.”

The APA’s review, like most violent video game studies to date, did not consider competition as a factor before drawing its conclusions. They even admit as much in the study: “Competition, then, may provide an additional independent influence on aggressive outcomes after playing aggressive video games. The literature on competition as the underlying causal component of the apparent link between violent game use and aggression is still nascent and is not currently substantial enough to influence, on its own, an objective assessment of the broader violent video game research.”

Makes sense, right? What makes you angrier: dying to a horde of violent aliens in Gears of War, or losing a close match to your taunting brother in the very non-violent Mario Kart?

It’s all of these questions—and the subjectivity of scientific studies, most of which can be used to draw any number of conclusions—that have convinced me to avoid reporting on these violent video game journals every time we get a new press release or meta-analysis. There just isn’t enough research or proper methodology to draw much from most of this science. The next time you read anything about the links between video games and aggression, keep all that in mind.

You can reach the author of this post at [email protected] or on Twitter at @jasonschreier