hypios

How many piano teachers are there?
I started taking piano lessons at age five.  My parents tried everything to get me to practice, especially appealing to my pride.  ”How many people can play the piano?” my father would ask.  ”You’ll be glad you learned as a kid!  How many adults are there who started taking lessons and wish they had continued?”  He would then start jotting down figures—how many piano teachers in our town, how many students each teacher had, how many would probably quit once they went to college—and come up with an estimate that proved I would grow up to be part of a very special and elite group of pianists.

My dad has long since given up on me as a pianist, but his questions still plague a different select group: search developers.  The future of search may lie in the answers.  In this two-part series, I’ll examine how search engines might answer them through social and semantic improvements.

Different ways to ask the same question
These are the kinds of questions we ask every day.  But Internet search is still not advanced enough to answer them.  If I want to find out how many piano teachers there are in the U.S., I have a few options.  Right now, search is split between giant search engines, like Bing or Google, and niche sites, like YouTube or Wikipedia.  On a search engine, I have a choice between using natural language (“How many piano teachers are there in the U.S.?”) and keywords (“number of piano teachers in USA”).  (A fun Slate article suggests that I also have a choice between “less intelligent” and “more intelligent” queries, i.e. a properly phrased question and “What is up with piano teachers?”)  I could also search for the websites of professional associations of piano teachers and look for my answer there.  On a site like Wikipedia, I could search for related articles: about the piano, about teaching music, or about American pianists.

Recovery vs. Discovery
Each approach, however, requires me to make selections and retrieve information.  Search results are in the form of links or articles, but what I’m looking for is an answer—in this case, a number.  Right now, search is excellent at recovery missions, like returning a piano teacher’s phone number or the name of a professional association of piano teachers.  This is information that you’ve either seen before or know exists; all you need to do is retrieve it.  Search is less helpful with discovery, when you don’t quite know what you’re looking for or where to turn.  Maybe you’re seeking simple sheet music for beginning pianists, or a good recording of Chopin’s nocturnes.  Search engines could improve discovery by indexing real-time content and returning results from social networks, leading to aggregated results.

Earlier search engines (remember Ask Jeeves?) focused on providing answers, only to be surpassed by keyword-focused platforms.  Instead of typing in questions, we’ve had to learn to phrase our queries as strings of keywords.  As the Internet has evolved, we’ve adapted our searches to take advantage of it.  One 2008 ReadWriteWeb article suggests that many children are more likely to use YouTube than Google for some searches, even though Google owns and displays results from YouTube through its ‘universal search.’  Kids can type in a keyword, and a how-to video or explanation will appear.  Just as we know which friend to ask for financial advice and which to ask about the best sushi places, we know which sites will answer certain questions better than others.

Search engines as your best friends
The question is whether search engines will eventually become smart enough to make these distinctions for us.  In a 2008 blog post, Googler Marissa Mayer describes the ideal search engine as a “best friend with instant access to all the world’s facts and a photographic memory of everything you’ve seen and know.”  Everything—and everyone you know.  The search engine as ‘best friend’ implies that ideally, it will be something to which you have an almost social connection, and that knows your social network.  Right now, there’s a wall between social content (the huge amounts of personal information on Facebook, for example) and the rest of the Web; this will have to come down if search engines are going to give ‘ideal’ results.

And it may be falling, albeit slowly.  In late October, Google and Microsoft announced deals with social networks Twitter and Facebook.  Bing will display results from public profiles on Facebook, and both Bing and Google will return relevant Twitter updates.  Google and Microsoft’s willingness to negotiate with others shows that the future of search probably lies in big search engines that do everything rather than niche sites.  (Look no further than this clever video to see how ‘Google’ has become synonymous with ’search.’)

How this will improve your searches
The deals will improve Bing and Google searches by allowing them to include up-to-date information in their results.  If you search for your favorite sports team, for example, you might see tweets from the stands at the latest game or Facebook exclamations about an amazing play in addition to the official score updates and links to the team’s website.  Even if the result is just a tweet linking to an article in The Economist, the article comes with a social context—it’s been voted up, essentially, by the Twitter user who decided to share it with his or her followers.  Instead of popping up in your Facebook news feed, though, the result will be displayed by Bing or Google, and it won’t necessarily come from someone you know.

Introducing the “social relevancy rank”
Does this mean you’ll get a flood of useless information from strangers?  Probably not.  Given that most people only look at the first page of results, search companies can’t expect users to wade through thousands of irrelevant status updates about other people’s children or pets.  Alex Iskold of ReadWriteWeb predicts that your results will eventually be filtered by a social relevancy rank, just as Google currently orders results based on its Page Rank formula.  (This is already possible, to some degree, on sites like FriendFeed, so the algorithms already exist.)  Rather than a stranger’s tweets about your sports team, for instance, you’d see a friend’s update about scoring tickets to a future match at the top of your results.  Search engines won’t just pull content from social sites; they’ll pull content specifically from your social graph.  Social indicators could also help with disambiguation—if all your friends are anthropologists, you’d probably like to see information about Claude Levi-Strauss, not read about Levi Strauss’s jeans—though context may prove more helpful.  You’ll see better results based on your network.

Should you make your profile public?
Of course, this means that search engines will need access to your social graph.  They won’t just pull status updates from your Facebook account; they’ll need access to your friend lists in order to analyze how you’re connected to other people.  Peter Mika, a Semantic Web researcher, has also suggested using object-based sociality to determine proximity—establishing “networks of instance, with associations showing the number of people who have tagged a given pair of instances.”  Put simply, commenting on the same blog post as a stranger implies some proximity between us, because we both read the article and were intrigued enough to leave comments.  And Google’s Social Graph API uses existing public connections to identify friends on different networks.

As a commenter on Iskold’s article pointed out, however, we understand that social proximity does not always equal social relevance.  You may be ‘friends’ with both your spouse and some kid you knew in high school on Facebook, but you wouldn’t give their opinions equal weight.  If search engines could weight opinions, they could display aggregated results, making them a trusted source for answers and opinions along the lines of Mayer’s ‘best friend.’

The question is whether we’re willing to let search engines access this information about our social lives.  Facebook is encouraging users to make their profiles more public, but the danger for the company is that users will remove most personal information before doing so.  Perhaps we’ll have to sign in to search engines to see personalized results.  Or perhaps, as Kevin Kelly argued in a 2007 TED talk on the next 5,000 days of the Web, “total personalization is going to require total transparency,” and privacy is the price we’ll pay for advanced Web features.

There are plenty of possibilities for search, and thousands of start-ups eager to improve it.  Given the Twitter and Facebook deals, however, the future of social search currently lies with the giants.  They’re only getting bigger and stronger…and maybe a little bit friendlier.

Stay tuned for a look at the future of semantic search in Part II!

(Answer to the piano teachers question: nobody really knows.)

Pic via Popdose