Geeks, Tweets and Cash
A conversation with Riley Crane of MIT Media Lab
Interview by Vikram Mahidhar > Photography by Matt Lennert
Twitter has permeated business, politics and mainstream media. Facebook has graduated from college and become a virtual social hub. More broadly, social media have emerged as a key boardroom discussion topic; there is a vague sense that something important is happening.
Yet for business leaders several fundamental questions remain around social media and more broadly around the use of online social networks in enabling business models.
- Do social media tools – and, more broadly, social networks – allow people to improve the way they perform their work?
- How do social networking tools differ from one another?
- What does it take to execute social networks-based strategies? How does their impact relate to their cost?
- Are there instructive examples?
Among the most prominent researchers in the emerging field of social media is Riley Crane, perhaps best known as the leader of the MIT Red Balloon Challenge Team that won the DARPA Network Challenge. In December 2009, by using social networks and tools, a group of researchers at the Massachusetts Institute of Technology edged out about 4,300 other teams in a Pentagon-sponsored contest to correctly identify the location of 10 red balloons distributed around the United States.
Dr. Riley Crane holds a Ph.D. in physics. While studying exotic properties of superconductors he became very interested in human systems and discovered ways of harnessing the power of social systems to create value. Crane is a member of the board and the inspiration behind a company called Charity Note that effectively uses social networks with the aim of collecting billions of dollars for charity from the annual $5 billion leftovers on plastic gift cards. He is a member of the Human Dynamics group at the MIT Media Lab studying complex social, economic and technological systems.
Crane was the 2009 Society in Science - Branco Weiss Fellow, with a focus on investigating collective behavior and prediction of human activity using ideas from statistical physics and fluctuation phenomena.
Vikram Mahidhar: Tell us about the DARPA Network Challenge. What were the key objectives?
Riley Crane: The Defense Advanced Research Projects Agency (DARPA) marked the 40th anniversary of the Internet with the DARPA Network Challenge (the Red Balloon Challenge), a competition that explored the roles the Internet and social networking play in timely communication, wide-area team-building, and urgent mobilization required to solve broad-scope, time-critical problems. The challenge was to be the first to submit the locations of 10 moored, 8-foot, red weather balloons at 10 fixed locations in the continental United States. The balloons were located in readily accessible locations and visible from nearby roads. Teams had to use the Internet to track down the locations. The challenge offered a $40,000 cash prize to the winning team.
VM: What differentiated your winning team from others? What role did social networking tools play?
RC: Not knowing fully what approaches the other 4,300 teams deployed it is difficult to identify the exact reasons for differentiation. However I know exactly what we did really well. We built a very powerful social networks-based strategy, what we call a “viral collaborative incentive program”, which mobilized crowds and helped the crowds and us to perform activities very efficiently and effectively to solve this big problem. Social networking tools ... enabled timely information sharing and collaboration, but it was just one piece of the puzzle.
In this competition we mastered a few other elements that I think are critical to deploying social networks-based strategy. These elements include:
- A clear, unified goal.
- A transparent incentive system.
- Analytic tools for timely analysis of qualitative and quantitative data.
- A feedback mechanism into the network to further guide crowds and facilitate collaboration
- Identification of reliable nodes—individuals—and building trusted relationships with them.
- Adapting and modifying our strategies and the direction of the network on the fly.
VM: It seems like there is a lot of confusion between the terms social networks and social media. These terms are frequently used interchangeably.
RC: There is a lot of confusion around these terms, especially in the business world. Let me begin with social networks. It is a group of individuals connected with each other or to a group based on interconnections or commonalities, such as relationships, interests, belief, profession, purpose and several others. Social networks have probably existed since the beginning of mankind; however they have become very complex and more visible over time. Social networking is interactions amongst individuals or groups within the social networks.
Before the advent of the Internet, individuals primarily networked in person at physical spaces such as offices, schools, community centers, and via ... mail and phone calls. The Internet has given a completely new shape to social networking, which has no limitation where you connect from, to how many you connect, and at what time. In other words, Internet-based social networking is flexible, scalable and instantaneous, which powerfully enables timely communication and facilitation of activities for the crowds.
Social media is a mode of mass communication that is based on social networks. Social media differs significantly from other forms of media in a couple of ways. One, it is not only a method of disseminating information but also a source of generating information through individuals who are part of the networks. Two, it enables many-to-many communication, back and forth, differing from the traditional broadcast format.
Now broadly speaking, there are two fundamental mechanisms that drive communication through online social networks. The first type is an endogenous mechanism – that’s my geek definition for “word-of-mouth” [laughs]. An example of a social media tool that enables endogenous communication is Facebook. In this system individuals are connected to each other in closed group fashion, such as with friends, friends of friends, family and other individuals that they are familiar with. The information is generated, disseminated and consumed by individuals within the closed group.
The second type is an exogenous mechanism. An example of the social media tool that enables exogenous communication through social networks is Twitter. In this system, individuals are connected to each other in an open fashion, beyond their personal circle. You can connect to anyone on Twitter, even if you don’t know them, and consume the information they generate. You can also generate information and make it available to anyone who is out there on Twitter.
VM: Most online social networks seem to be geared towards recreation, and active participation of individuals in these networks is generally spotty. Does having a clear goal improve participation?
RC: It is a myth that social networks are primarily geared towards recreation. There are various types of social networks aligned by common goals or a purpose. Some of these network types include professional, educational, public services and community connections. There are hundreds of online social networks out there that have a clear purpose or goal. And there are hundreds of online social networking tools that are targeted at facilitating networking around focused goals and purposes. For instance in the health sciences industry there are over 200 online social networking tools that connect patients, doctors, life sciences companies and regulators with goals ranging from dealing with a rare disease to conducting clinical trials.
It is very important to have a clear goal or a set of few goals for a social network to effectively create tangible outcomes. A clear goal provides focus and aligns crowds to conduct activities in line with the desired outcome. It is similar to the need for clear goals for an organization to be successful in achieving what they aspire to be. Also, it creates a natural filter for the network participants and attracts more relevant individuals. Such individuals are more likely to contribute in a meaningful way.
Just having a clear goal does not assure that everyone will fully participate; it is one important element. To motivate participation I think it is important to have other elements in place including an incentive system, analytics and a feedback mechanism.
VM: Is social networking effectively free, given that you don’t really have to hire people to get things done?
RC: No, the social networks approach is not at all free. Today many organizations put the social networking infrastructure in place in anticipation that “once it is built people will come and deliver.” Such programs fail horribly because companies do not invest adequately in managing and developing these networks, which is resource intensive. Building momentum across participants, keeping them engaged and steering the crowds in the right direction requires a lot of input into the network on an ongoing basis. To conceptualize and implement such strategies requires a wide set of capabilities including human dynamics, behavioral economics, data analytics, systems thinking, deep knowledge of the problem set being put in front of the network, operations management, risk management and technology. Each member on our team brought one or more of these capabilities to the table, which is one of the secrets to our success. It is certainly a faster way of engaging scores of people to get things done.
Additionally, use of social network applications in business or operating models is still in its infancy. We don’t have many proven models yet. Organizations that got it to work have also allocated significant resources to trial and error before they got it right.
Depending on the industry and the goals of deploying this approach, organizations also need to invest in managing business and regulatory risk. Most of the regulators are still mulling over the guidelines for the use of social networks. Hence the organizations need to be proactive in developing their own governance guidelines.
And finally, it should not cost individuals much (nothing if possible) to participate in these networks. There should be a clear incentive system in place to motivate participation of the right people. Again, this does not come for free.
VM: What did the incentive system look like in your social networking approach? What types of incentives are effective?
RC: Incentives drive behavior. In our approach we wanted to encourage individuals to help us cast a very wide net of individuals across the country through word of mouth, with the goal that some of those individuals should be able to find the true locations of those 10 red balloons and ultimately pass that information back to us in a timely fashion. The incentive system that we built offered rewards for everyone across each of the 10 chains of individuals that led us to the true locations of the ten balloons. Since every individual in the network stood a fair chance of winning some reward, irrespective of them finding the balloon themselves or not, it encouraged individuals to collaborate with each other with a common goal of getting closer to the location of the balloon. So there are two takeaways here. First the incentive system should be designed to drive collaboration. Second, it should encourage individuals to get closer to the goal.
Another takeaway is that it should be transparent. On our website upfront we clearly articulated how the incentive system will work, which helped individuals build confidence in our team. I shouldn’t discount the fact that very high reputation of the MIT brand associated with us also helped build confidence in our team.
In addition to using the publicly available social networking technologies such as Facebook, we built our own IT infrastructure that helped us track activities down to the individual level and across the chains. We used this infrastructure to create accountability and transparency in the incentive system. The IT system also enabled us to connect directly with each of the individuals, which in turn allowed them to send the relevant information to us directly without losing any time by sending it up the chain. The take away is to create a robust infrastructure that ensures accountability of activities and transparency into the incentive system.
With regard to the types of incentives, this is a field of study in itself. While I have not studied this area exhaustively, I can share some of our experiences and my knowledge about what others are doing. This topic is of interest to almost every Internet-based company from an online retailer to web search providers to gaming companies, who are trying to harness wisdom of crowds. As Nicholas Carr and Clay Shirky describe in their work, the online crowds perform four to five key functions. At a high level these functions include lending their distinct talents to create a product such as Wikipedia; acting as a survey group providing average opinion on a issue; acting to produce a set of behavioral data; simple information trading; and instigate and coordinate point-to-point transactions.
If we explore the incentive systems underlying these functions, I believe they probably are distinct but generally some combination of both financial and non-financial incentives. In our case we offered both financial and non-financial incentives. It was publicly known that the winning team will get a cash prize of $40,000. More than just the financial incentive, I think our willingness to share all of what we were going to earn with them reinforced their trust in us. In addition, in an after-the-fact survey of the participants, we identified that our announcement around the non-financial goals, which was to contribute our learning to further advancement of sciences and to give away the remainder of the money to charity, significantly motivated the crowd.
Non-financial incentives are used more widely. Non-financial incentives used in online social networks are generally geared towards helping participants build their social capital and helping participants achieve their personal goals. The incentives are designed in many different forms. Some of the popular forms are awarding recognition and status based on the participants’ contributions and qualifications. Also some networks are designed and branded to provide a sense of belonging or affiliation, which are great motivators.
While designing the incentive systems for online social networks one should ask two fundamental questions: Why would an individual participate? And why would they get anyone else to participate?
VM: Large social networks produce huge amounts of unstructured, qualitative information every day. How useful is this information, and what type of analytics capability is required to mine it?
RC: Most of the large online social networks generate terabytes of data every day. For instance, Facebook manages more than 25 terabytes of data per day in logging data, which is the equivalent of about 1,000 times the volume of mail delivered daily by the U.S. Postal Service. Mining this data to use it meaningfully is the single biggest challenging task.
Analytics is at the heart of orchestrating social networks and steering the network towards a goal or a purpose. Even though the objective is to harness the wisdom of crowds, generally the crowds do not directly share wisdom. They usually share information and knowledge, which requires validation and in many cases analysis to develop insights and wisdom. While the basic premise of social networks is to use humans to validate the information and collaboratively solve problems, these activities need to be monitored and analyzed in the background for several reasons.
The first reason is to filter the “power” participants from others. The objective here is not to drive other participants out of the network – they also play important roles. However it is to identify the participants who demonstrate a very strong, relevant network and the potential of contributing in many different ways so that they can further empower and influence the overall network activities and relationships. In addition, the information flow from such participants can be more valuable so it can be analyzed with more rigor.
Next, analytics can be used to validate information and feed insights back into the network. Analytic models are used to validate certain types of information in almost real time to ensure that the crowds do not waste their cycles by building on to incorrect information. In the case of the Red Balloon Challenge, while humans in the network acted as a binary sensor to validate information, there is always slippage, and it is extremely useful for the network when the correct and incorrect information is distinguished for them and highlighted. For example, during the Red Balloon Challenge we used simple IP filtering to match the location of the information provider and the information that they provided on a balloon’s location. If the IP address of the information provider was not within the 20 mile radius of the balloon location that they provided, we flagged that information as invalid. It allowed us to filter almost 90 percent of the responses, allowing the network to focus on the right ones.
Additionally, injecting useful insights from the analyses back into the network can significantly enrich the knowledge base of the network. Generally network orchestrators are the only ones that conduct collective analysis of this huge information flow and can connect the dots to develop unique insights and guide new ways of thinking. It also keeps the momentum going in the network. For example, initially we did not know that the DARPA was providing exact geographic coordinates for the balloons. When we found this out from one of the entries, we instantly communicated it back to the network which further helped them refine their searches.
Finally, analytics can help us understand behavioral patterns to adapt the model and to manage the risk. Every social network is unique in some ways. It is hard to exactly model an approach. Also the networks are dynamic, so the models need to adapt from time to time. Behavioral analysis provides insight into how people respond to a certain type of information and where the nucleus of activities is. Such insights can be the basis for adapting the models.
VM: How did the social networks approach allow your team and other people in the network do their job better?
RC: We could not have solved this problem in 8 hours and 52 minutes without the social networks approach. In fact, the social networks approach allowed us to deliver on all three metrics - better, faster and cheaper. Finding the 10 balloons dispersed across three and a half million square miles [total land area of the United States] was like finding a needle in a haystack. While we are testing which algorithms and network analysis would have been effective had they been in place on the day of the challenge, much of our success came down to traditional methods of investigative journalism. Moreover, even if we had sophisticated algorithms in place, it would have been very costly and would have taken days if not months to analyze the results. Through the online social network we were able to benefit from the talents of hundreds of people and their knowledge to very quickly narrow down a few potential locations and ultimately find all 10 balloons. Our fundamental operating principle was that it should not cost the participants anything to contribute, which definitely made this effort very cost effective.
VM: What are some other successful examples?
RC: There are several well documented examples of successful social networks approach to problem solving. One of my favorite examples is Ushahidi-Haiti, a crisis mapping platform based on social networks that very successfully enabled many global organizations to rapidly and effectively respond to the recent Haitian earthquake crisis.
Ushahidi provides an open-source, free service which can overlay maps of affected regions with data gathered from a wide variety of sources including social networking sites, email, news sites, blogs and mobile text messages. Detailed maps can show, for instance, where the people are trapped and where the safe drinking water is available – as in the case of Haiti. Ushahidi enables a powerful crisis management infrastructure through engagement of crowds. It connects people who have been impacted by the crisis to the relief providers and to the network of volunteers who help validate the information and mobilize resources to support relief.
In the case of the Haiti crisis, within two hours of when the aftershocks were felt, the Haiti site was launched on Ushahidi. Within a few hours, Ushahidi mapped 2,900 reports from Haiti on their site. Soon after, many global relief organizations such as the Red Cross, U.S. Marine Corps, the Federal Emergency Management Agency (FEMA), the Office of U.S. Foreign Disaster Assistance (OFDA) and many others started utilizing Ushahidi as a key source of information for their rescue efforts.DR
Vikram Mahidhar is the deputy leader of Deloitte Research, Deloitte Services LP.