hypios

Jeff Howe’s Crowdsourcing Re-Visited
hypios has sometimes been called a crowdsourcing platform. We always felt like this comparison was missing an important point about hypios and crowdsourcing. So we took a closer look at the concept. A major contribution to the discussion surrounding crowdsourcing is Wired editor Jeff Howe’s book Crowdsourcing:  How the Power of the Crowd is Driving the Future of Business (2008). We took his contribution as a starting point for our own thoughts.

Howe’s book seeks to show the internal connection between a dispersed family of phenomena cropping up all over the web.  He urges us to see the profileration of wikis, a threadless.com T-shirt design contest, and the collective search for alien life via spare computer cycles as aspects of a single phenomenon: crowdsourcing.

Now one of Howe’s main examples, in working out the meaning and significance of crowdsourcing, is the (recently-concluded) Netflix Prize, which has spurred a lot of commentary here as elsewhere.  For those who don’t know, in 2006, Netflix offered $1 million to the first solver(s) who could improve their film recommendation algorithm by 10%.   The actual outcome of the Netflix challenge, coming about a year after the book’s publication, invites us to take a more critical stance towards some of Howe’s (widely-accepted) arguments.

What is crowdsourcing?
Crowdsourcing, according to Howe, is an umbrella term for diverse phenomena.  It can involve:

• Tapping spare time (or procrastination propensity) to rate content.
• Knowledge-transfer across disciplines or research silos.
• Innovation through incremental improvement, where the quality of the outcome is indexed to the sheer diversity of the solvers involved. (The Netflix challenge is one example.)

Diversity and crowdsourcing
Howe takes diversity as the key reason crowdsourcing is a superior mode of sourcing. And for him, diversity of solvers’ backgrounds and abilities is more important than anything else.  There are two levels of diversity that Howe sees as relevant to crowdsourcing:

• Diversity of fields represented.  Here Howe’s reference is primarily the work of Harvard B. Prof Lakhani, who discovered that those most likely to crack a thorny problem were solvers from a field peripheral to the problem’s domain. These peripheral solvers could easily see a solution because, in their field, the problem had already received a standard treatment or solution.  Diversity of solvers was critical for this sort of knowledge transfer.
• Cases where the solution to the problem does not involve a transfer.  If no solution exists, finding it will require a diverse set of solvers, not just from different fields but also of varying cognitive abilities.  This is probably the boldest of the book’s claims.

Team Mensa vs. Team Brown Socks: Howe’s view
The diverse crowd will almost always beat the team that R&D directors would likely choose themselves.  This is because the latter would always go for the “Mensa team” (the society for people with high IQs)—the team recruited from, say, the top 5% of MIT engineers.  According to Howe, this is because diversity in terms of distinction and ability will yield greater creativity in terms of analytic orientations.  Howe’s amusing image for stocking Team Diversity is randomly rapturing all the profs wearing brown socks from a mid-tier American university faculty lounge.   His Exhibit A for this type of crowdsourcing is the Netflix competition.

What  really happened
The only problem with Howe’s analysis is that it doesn’t describe what actually happened.  From Howe’s argument, it seems like the winning team should be a (relative) band of losers.

Howe’s analysis has suggested that, through collaboration, improvement would be incremental (all about the tweak) and come from the darndest place (e.g. unemployed psychologists), leading to a cinematic triumph of the underdog.  Instead, when you look at the two top teams, it ended up looking a lot more like a battle of the Mensa teams. On the one hand was Pragmatic Chaos, a joint effort of the two previously highest scoring teams from AT&T’s Belkor Labs.  On the other was ENSEMBLE, a mash up of lower-ranked teams. Members of the former write articles theorizing matrix factorization techniques for IEEE, while the latter, judging by its roster, is nothing but a giant geek-agglomeration, stocked full of well-credentialed computer scientists and statiscians.  Not so “brown socks” after all.

Team-building and networking are more important than diversity
So what happened to the diverse crowd? In fact, the way hypios sees it, the right way to think about crowdsourcing is less Howe’s  ‘it takes a village’ than it ‘takes a well-developed network.’  (We’ve already argued for this here and here). Better networking and collaboration tools allow the strongest solvers and teams of solvers to self-identify out of the crowd.  Theories about crowdsourcing have correctly emphasized the fact that a marketplace for ideas is essentially social; however, they tend to over-emphasize the democratizing aspects. Crowdsourcing is an alternative, and superior, method for identifying exactly what Howe calls the MENSA team—highly-qualified and highly-trained specialists—by gleaning them from a much larger crowd.

According to Netflix stats, it took three years, over 44,000 valid submissions (from 180 different countries) and 5,169 different teams to best the CineMatch algorithm by a little more than 10%. Clearly, the most notable trend is that participants chose to collaborate by merging into larger and larger teams. As one PhD solver notes, the contest was all about team “agglomeration” where yesterday’s top losers banded up to tag-team today’s winners.

This capacity for iterative-rounds of team building, rather than the diversity of solvers, is probably the factor to isolate when we think about why various types of crowdsourcing work.  Howe is surely right when he says that crowdsourcing yields a superior, more diverse team.  The team is not superior because of its diversity, though, but because it was assembled in the course of the problem-solving process. This allows the problem itself to determine the competency profile of the team needed to solve it, not some ousider’s ideas about what competencies will be needed.

Crowdsourcing actually identifies the best people
If the Netflix contest actually created a super-mechanism for creating self-selecting Mensa teams, then what Howe is describing looks less like a hierarchy-demolishing, collaborative mode of production and more like the new type of market mechanism that hypios’ platform represents—one that intensifies competition in ideas. Howe himself emphasizes that embracing “crowdsourcing” involves embracing the principles that “the best people are always working for somebody else” and “you don’t know who they are.”  What he doesn’t seem to want to say, however, is that crowdsourcing is in fact a new (and surprisingly direct) means for identifying who and where these best people are.

Debbie Goldgaber