We've all seen their work. "Analysts predict PS3 price drop by September." "New DS to be released by Nintendo." "Wii will be the top selling console of 2007." Analysts seem to descend from their mountaintops once a month to tell us the future of the video game industry. So what secret rituals do these soothsayers conduct to get their predictions? And, more importantly, how often are these predictions right?

After a look at ten analysts, 100 predictions, and first hand accounts from video game industry analysts Michael Pachter of Wedbush Morgan Securities and Jesse Divnich of The simExchange, and founder of The simExchange Brian Shiau, we managed to gain a little bit of insight into the murky crystal ball used to prognosticate and pontificate.

So what is an analyst, anyway?
Video game analysis is big bucks. Stocks in gaming companies are bought and sold every day, and traders rely on the advice of analysts of what stock to buy, since many of those who trade stocks don't know much about the game companies or consoles themselves.

"Because most investors can choose from 12,000 different stocks, few are very well-versed in any specific stock," Pachter said. "Equity analysts cover around 15 stocks, and because the number is limited, the analysts are able to know everything about the stocks that they cover. That makes them 'experts', and their knowledge is valued by institutional investors."

Traditional analysts such as Pachter collect all the information they can about the companies they cover and their products, quantify that information through number crunching and spreadsheets, then present models and predictions to their clients in meetings and through reports. These traditional analysts are more concerned with companies as a whole, not with specific games. Games are only relevant in terms of how much revenue they will generate for a company.

A variation on this kind of analyst is the research analyst, like Billy Pidgeon. In a GameTrailers TV interview with Geoff Keighley, Pidgeon explains that as a research analyst looks at the historical sales data of consoles and games, and from that makes predictions about trends, the install base for consoles, and increasingly the future of online gaming. Research analysts compile this data into reports, which are then sold to other analysts. Pachter says that the audience for a research analyst is anyone who is interested in the video game industry sales, while his audience is primarily investors.

The simExchange works differently, and instead uses a prediction market. Divnich bases his forecasts for the industry on the trading of fictitious stocks in games and consoles by over 8,000 members of the website. Stock traders of The simExchange base their decision on three video game factors: global lifetime sales, US monthly sales, and metacritic rating, which summarizes how favorably the game reviewing community received the title. Anyone can join The simExchange.

"It looks like it may be a bunch of people who aren't very informed," Brian Shiau of The simExchange said, "but what happens in the prediction market is people are putting their money where their mouth is."

The main qualifications necessary to become a more traditional analyst are mostly business-based and not so much about video game experience, although a healthy appreciation for gaming definitely helps.

"At a minimum, most equity research analysts have either an MBA or a CFA designation," Pachter said. "Beyond that, there are no hard and fast qualifications."

Reputation and accuracy in fact seem to be what matters most in becoming a good analyst.

"Believe it or not, the qualifications are pretty low," Divnich said. "You just need to have a strong understanding of both the video game industry and financial markets. If you have that, it simply takes a lot of hard work to prove yourself among your peers and if you are truly good at what you do, advancements will fall in your favor."

What we wanted to find
With analysts handing out predictions about every gaming system and many of the top games, you have to wonder how often they actually get it right. We did. So Kotaku went on a mission: Find ten analysts, find predictions for each of them, and prove them right or wrong. Simple, right? Actually, easier said than done.

While some analysts make concrete predictions, such as how much money a game will make in the US for a specific month, other analysts seem to dwell more in the realm of hazy, undefinable predictions. This latter group make predictions that are very general, or they make predictions that won't be provable for another few years - at which point, their prediction will probably be long forgotten.

For the study, we looked at ten different analysts:

• 
  
• - Michael Pachter, Wedbush Morgan Securities
• 
  
• - Colin Sebastian, Lazard Capital Markets
• 
  
• - Piers Harding-Rolls, Screen Digest
• 
  
• - Jesse Divnich, The simExchange
• 
  
• - Evan Wilson, Pacific Crest Securities
• 
  
• - Mike Hickey, Janco Partners
• 
  
• - Jeetil Patel, Deutsche Bank
• 
  
• - Mitsuhiro Osawa, Mizuho Investors Securities Co.
• 
  
• - Richard Doherty, Envisioneering Group
• 
  
• - Billy Pidgeon, IDC
• 
  

There are many more analysts than just these guys, but all of these analysts made about ten predictions pertaining to the video game industry which were provable as right or wrong. There were a few exceptions: Harding-Rolls and Pidgeon both have quite a number of public predictions under their belts, but most of them are too far out to see if they're right or wrong yet. Harding-Rolls has been a video game analyst since 2005, but many of his forecasts won't be provable until at least the end of this year - some not until 2011. As for Pidgeon's predictions, many of his are about online gaming, for which there is limited data, other than the statistics quoted by Pidgeon in news articles. Online gaming revenue stats aren't as readily available as the ones for software and hardware, but presumably, as online gaming is becoming more widely recognized as a huge moneymaker, these stats may become as easy to find as NPD numbers.

Osawa also proved an interesting anomaly for the group, as he does not cover video game companies exclusively, like a number of other analysts. He didn't make ten predictions about video games specifically, but he did make predictions about other technologies, including televisions, digital cameras, and photocopiers.

The Results

No analyst was completely wrong, but no analyst was completely right, either. Billy Pidgeon was the lowest scoring of the analysts, with only one out of seven of his provable predictions correct (meaning even if all of his predictions were provable, he wouldn't be able to score higher than four out of ten predictions right, still at the low end of the results).

The highest-ranking analysts (still with only six out of ten) were Pachter, Patel, and Osawa, but that doesn't necessarily mean they were the best at predicting overall. Both Patel and Osawa made very generalized predictions that had a greater chance of being correct. For example, while some analysts gave dates for when they suspected the PS3 price cut was coming, Osawa simply said that a price cut would be one option to help the system's sales. Likewise with Patel, who said the price cut wouldn't happen until at least April 2007, meaning the July 2007 price cut the PS3 saw still qualified Patel's prediction as being true.

The one surprising exception to this rule was Pachter. Pachter's predictions were very specific, such as pushing GTA IV's release back by a year, and John Woo wanting to make more video games. But being right most isn't what Pachter says matters most to him.

"None of my predictions are correct, but most are close," Pachter said. "The nature of my work is like a game of horseshoes, where close counts."

Divnich also says that the exact accuracy of predictions isn't what really matters in the video game prediction business.

"There really is no good percentage, just try to be more right than your peers," Divnich said.

Divnich also explains why some of the predictions are so hard to prove true and false.

"You really can't put predictions in those terms since most of the time we are predicting a series of events and not just numerical predictions," Divnich said.

The events that these analysts predict on are generally fairly common. One of the more popular predictions was when the PS3 price drop would happen: Michael Pachter, Jesse Divnich, Evan Wilson, and Mitsuhiro Osawa all voiced an opinion on when and whether or not the PS3 would drop in price. Pachter's prediction of a price drop in the 80 GB model as supplies of the 60 GB model dried up came true, while Wilson's suggestion that the price cut of the PS3 wouldn't happen any time soon was made just four days before Sony announced the price drop on July 9.

Most of the predictions analysts make are about the NPD sales data that gets released each month, forecasting how well they think the companies in their portfolio will do. Hardware sales are of particular interest to analysts, so industry changes that may affect the console war, such as many studios going Blu-Ray exclusive, will impact what an analyst thinks of a product and it's company.

What's most interesting about all the predictions, though, is that the best analyst isn't necessarily the person who makes the most correct predictions. Some analysts make more complex predictions than others, so it's more impressive to see an analyst who is only short by a little bit on their numeric predictions than an analyst who is correct more often on very general predictions. In terms of the worst, analyst Evan Wilson made predictions that were either very general and easy to call, or outlandish and would be shocking if they came true. In November 2007, Wilson claimed that Nintendo had a new DS redesign ready for when the sales of the DS Lite began to taper off. Nintendo quickly quashed this claim, leaving Wilson out to dry. Wilson also claimed that EA would drop the price of 14 of its sports titles over the holidays, but once again, there was no truth to this prediction.

One of the best analysts overall turned out to be Colin Sebastian. While he only got 4 out of 10 predictions right, he made strong predictions with figures, many of which were quite close. He predicted that Activision's Q3 sales figures would double, and was very close (going from $142.8 million in 2006 to $272.2 million in 2007). He predicted that there would be a shortage of Xbox 360s over the holidays, but it turned out the effect of that shortage didn't turn up until January-February of this year. He predicted that Halo 3 would generate $200 million very quickly, and while it didn't reach that number on the first day, it did hit $300 million by the end of the week. If being close is the name of the game, then Sebastian is certainly one of the top analysts to watch.

What's in it for me?
So if the predictions are mostly for people with stock, and the analysts don't think it matters if they're particularly accurate, what are gamers supposed to get out of all these analyst predictions?

"Global lifetime sales seem to interest a lot of gamers," Shiau said. "We find a lot of gamers care about how the product sells, even if they aren't investors, or working in the industry,"

Beyond having a passing interest, there is one major reason why gamers should be interested in these results.

"You don't need analysts, but investors do," Pachter said. "The companies that make the games [Kotaku] readers enjoy need capital in order to develop those games. Investors provide that capital. Without analysts, there would be lower demand for video game stocks, share prices would be lower, and compensation would be lower for the developers. That means lower quality games, and [Kotaku] readers would be unhappy. So in a perverse way, equity research analysts make it possible for publishers to make great games."

At the end of the day, gamers shouldn't be relying on analysts for all their video game information. But they also don't need to. Analysts are there for one reason: money. Most gamers have an interest in how well games do because, like Pachter says, without funding great games won't be made. However how much money a game makes won't change whether or not it's fun to play: that's always left up to interpretation.

Check back Friday for a full break down of the 100 predictions and how they fared.