hypios

We’ve been watching, over the years, a certain contest with great interest.  Netflix is an American DVD rental company that revolutionized its industry with the simple idea of letting people select and order films online and then receive them in the mail. It was a huge hit.  What they always wanted to do better, however, was predict the kinds of movies that their customers would like, based on other films they’ve seen and the rating they assign it (customers are sent a brief survey to evaluate recent rentals), in order to push films towards customers that they’re more likely to enjoy.

This search for a predictive algorithm is the first reason for our interest, and we are not alone. Predictive algorithms are on many companies’ wishlists. Now that we have access to all this data, like social network profiles that list a user’s preferences and interests (plus the site’s internal trackers that record users’ behavior patterns), the thought is that this massive amount of data should allow us to predict what someone will like or dislike, purchase or ignore.  For example, hypios wants to be able to push problems to our solvers that they would be likely to have the ability (or desire) to solve, or just know someone in their network who might.  However, to be able to predict these sorts of things, as the Netflix contest aptly demonstrates, is massively complicated.  We’re far from mastering it, though it’s possible that Netflix brought us closer.  We look forward to learning more about the kinds of strategies that were used by the teams to come up with the winning algorithm.  The awards ceremony is set to be held in September. (For now, this video to see what some of the solvers have to say about their strategies.)

Equally of interest to us are the dynamics of the contest itself. Netflix decided to offer $1 million for anybody who could improve their existing predictive algorithm, CineMatch, by 10%.  They broadcast the problem to a group of solvers attracted by the prize money Obviously, we at hypios wanted to know how it went, since part of what hypios is designed to do is generalize the principles behind the Netflix challenge.  Here’s what’s interesting about the dynamics of the Race.  While (sadly) the euro value of the prize has fallen during the three year’s since the competition’s début, the response of participants was: “No worries.”

Apparently, they really weren’t in it for the money—the competition’s appeal was the chance to compete, collaborate, and the pleasure of solving problems.  The money was just an excuse for the fun, it seems.  This confirms one of the driving presuppositions we’ve had here at hypios: while getting paid is great, friendly competition might just be the motor of it all.  The challenge is always to think about how to create the kinds of mechanisms that will make this friendly competition more challenging and rewarding to solvers.

One surprise, even to Netflix, was the spontaneous decision by teams to share aspects of their solution with each other.  (And this is with a million bucks at stake.)  It might be that participants realized that the problem was simply too hard to solve alone, and after a few rounds it was clear to all the solvers that sharing was the only way to move forward.  If the solvers bet that reaching the 10% threshold was going to be a matter of incremental innovation (barring a cognitive leap among one of the solvers) then the game transformed itself to a round of hot-potato (except, in this case, whoever had the potato when it reached the 10% could exchange it for $1 million).

The cognitive leap never came (which is why the end was a photo finish, with two teams in a dead heat) and incremental development ruled the day. One of the key points here is largely that it was up to the solvers to decide how they would interact—to create, as it were, their own game—and what they chose to do, was maximize the sharing/collaboration aspect.  Existing teams participated and participants formed teams, sometimes as a competitive response to whichever team was currently in the lead.  Indeed, the dynamics of the contest, when studied, are a game theory gold mine.