These days, Steam is one-part library, one-part labyrinth. The sheer volume of games on the service is overwhelming. Valve’s solution to the problem has been to double down on algorithms, but many still don’t feel like Steam is helping them find diamonds in the rough. Enter Steam 250.
Steam 250 is a site whose goal is simple: find the best games on Steam. It has a variety of lists, from the 250 best games to hidden gems, and everything in between. Its creator, Bilge, decided to create the site when he ran out of games to play. He realized that Steam tends to focus on games that are selling well or getting lots of attention and, in order to find hidden gems on Steam, he’d have to do a lot of digging.
“Steam tends to focus on sales data to promote games,” he said in an email. “Comparing our annual sales versus ratings rankings shows there is a wide disparity between games people are willing to buy and those they’re willing to recommend. Focusing on sales provides an incomplete picture of the gaming landscape and frequently misses games more gamers are enjoying.”
Bilge decided to make a solution: his own Steam algorithm. Steam 250 bases its selections on user reviews, specifically the number of reviews any given game has on Steam and the percentage of those that are positive. On Steam 250's website, Bilge explains that determining the exact way the algorithm weighs games with different numbers of players and reviews took quite a while.
“During early development of Steam 250, we trialed seven different algorithms, each with their own behavioural characteristics,” he wrote. “Each algorithm is further tuned by a weighting variable that biases the score in favour of either the number of votes or approval rating. Each algorithm’s weighting exists on an unbounded, sliding scale thus creating limitless possibilities. We narrowed down the possibilities to 22 distinct combinations of algorithms and weightings and scrutinized them until a winner could be declared.”
The end result is a ranking system that won’t, say, propel a game to the top just because it’s new and 100 percent of its two or three reviews are positive. It also allows for lists that point to games with relatively high numbers of positive reviews despite low player counts (with player count data provided by third-party Steam analytic service SteamSpy). Basically, if you’re looking to quickly cut through Steam’s clutter and find cool games you’ve never heard of, Steam 250 can help with that.
That said, there are still potential holes in Steam 250's methodology. For instance, while scanning the site’s various lists, I came across a handful of games that looked pretty shoddy, but were drowning in positive reviews because they handed out achievements like Halloween candy. Bilge acknowledged that things like that can happen, but his process is backed by data and data alone, for better and worse.
“It’s not our place to judge whether games have acquired reviews ‘fairly’ or not,” he said in an email. “In general we find the review system pretty reliable for pointing to good games. One could extrapolate that if a game gets a lot of positive reviews just because it’s an achievement farmer, it’s still a positive experience for everyone who wants to discover achievement farming games, although I admit that’s a little far-fetched and we wouldn’t want an endemic of such games trumping genuinely good titles.”
There are other possible issues, too, like review bombing, which even Valve has acknowledged—in its own, graph-heavy way—is a big problem. As it stands, a targeted review bomb or two could send a game plummeting from its deserving perch on Steam 250.
In the future, Bilge hopes to make Steam 250 more elaborate, with options to browse games’ history (which could help with the review bomb thing), search based on exact tags, and even modify the algorithm’s weighting yourself. Beyond that, though, he’s open to suggestions.
“The site began as just one ranking and has grown into more than 100 based on user feedback,” he said. “I expect it to continue to grow this way, responding to gamers’ needs.”
You’re reading Steamed, Kotaku’s page dedicated to all things in and around Valve’s wildly popular PC gaming service. Games, culture, community creations, criticism, guides, videos—everything. If you’ve found anything cool/awful on Steam, send us a message to let us know.