Making Sense of Social Data
Digital exhaust and the next frontier in social data analytics
By Doug Palmer, Vikram Mahidhar and Dan Elbert > Photography by Matt Lennert > Infographics by Ilovedust
In 2010, Google’s Eric Schmidt said that “I don’t believe society understands what happens when everything is available, knowable and recorded by everyone all the time.”1 He was referring to the fact that in the digital world, data are everywhere. We create them constantly, often without our knowledge or permission, and with the bytes we leave behind, we leak information about our actions, whereabouts and characteristics.
This revolution in sensemaking—in deriving value from data—is having a profound and disruptive effect on everything from supply chains to corporate strategy. In particular, it is generally forcing executives to rethink how they understand, reach and even influence their customers. Simply put, with so much data available, especially on social networks, the ability to know the people you sell to and to monetize that knowledge has never been greater.
That said, most companies are only beginning to scratch the surface of what’s possible with social data. Many are still operating in the pre-social media age, simply trying to make sense of the data they have—rather than the variety of sources that exist. But even those that are best-of-breed have only started to tap into the true potential of this information: developing an intimate and real-time knowledge of customers’ relationships and behavior.
"The next big data revolution will be around mobile. If you think about mobile devices as sensors, our phones know more about us than our partners do."
Whole new opportunities are available with these kinds of insights in hand. Banks can evaluate loan applicants based on the credit worthiness of others in their social network, the notion being that people who pay off their loans tend to group together. Telecoms can find their most influential customers—those who might switch providers and take a number of friends with them—and target them for early upgrades or deals. Even human resource departments can benefit, by understanding which applicants are most connected, or even most passionate, in a given field. The data points have typically been limited only by technology’s ability to capture and store them, a constraint that is rapidly fading away.
Still, very few companies are able to act on this kind of information, let alone accumulate it. However, as we will explore, what it takes goes beyond just an analytics solution.
Customer analytics is hardly news to most executives; organizations have been gradually embracing it over the last 20 years. Amazon.com was an early leader in this space, for instance, pioneering the art of using buyer behavior to personalize its site. For Amazon.com, no click goes unnoticed. Its model of the customer has very little to do with your name, age and address, and everything to do with what the company been able to learn about you by studying your data trails.
But even some organizations that long predate the digital age are masters at analysis. Credit card issuers, for instance, have for decades used data to predict everything from spending patterns to risky behavior. Continental Airlines, which was founded in 1934 and recently merged with United, is another example. For its most profitable customers, the company tracks every kind of “flight disruption” that might lead them to switch carriers. Whenever one of them is delayed, or if the airline loses their bag, cabin crews are automatically alerted the next time they fly.2
These examples highlight what we think of as the first of three waves playing out in the space of big data. In this wave, organizations can tap an incredible amount of information—purchasing histories, demographics, measures of engagement—that make customer targeting more feasible. They can use everything from clicks and flights to credit card transactions, made on and off the Web. The key feature of this wave, however, is that the data inside it are not social – they are drawn from closed or proprietary mechanisms instead of what is often stored openly on the Internet. More importantly, because these data aren’t social, they lack a broader context about the relationships and behaviors of the people creating them.
The second wave gets closer to adding that color and came about with the rise of social media. As Andreas Weigend, the former chief scientist at Amazon.com explains: “Users started to actively contribute explicit data such as information about themselves, their friends or about the items they purchased. These data went far beyond the click-and-search data that characterized the first decade of the Web.” Rather, they provided specific new insight into customers by putting their decisions in a social and personal context.3
Twitter, Facebook and LinkedIn are familiar platforms in this wave. Broadly speaking, their data are valuable because they tap into the voice of the consumer, telling us things like who they know and what they like. For companies looking to target ads or customize their products, this information can be incredibly powerful.
It can also drive new forms of digital listening. Many organizations, for example, are starting to build social media “command centers” to take in social data and react to them in real-time. Gatorade is one such example. Inside their mission control room, a team of five people use sophisticated sentiment analysis software to monitor and shape Internet conversations as they take place.4
However, the voice of the customer is by and large explicit. It is all tell, no show. If you have a Facebook account, for example, and visit NYTimes.com, you can see articles recommended to you by your friends. But few news sites, if any, actually analyze your behavior—or that of your network—to suggest content that is entirely new.
In fact, this kind of implicit analysis of social data is only beginning to emerge – and constitutes what we see as the third and most powerful wave of data analytics. In this wave, the focus shifts from the voice of the customer to the individual’s behavior. But more importantly, it looks at that behavior in the context of who is around them and how they interact. It gets below the curated, surface-level information we put out about ourselves online, to help understand what our digital trails and network really say about us.
Early examples of this can be found in large organizations, to be sure. We alluded to one earlier: several telecommunications companies use call data to find influential customers and target them for more effective plan upgrades or special deals. According to a recent article in The Economist, “Bharti Airtel, India’s biggest mobile operator, which handles over 3 billion calls a day, has greatly reduced customer defections by deploying [social network analysis] software … [and] IBM, the supplier of the system used by Bharti Airtel, says its annual sales of such software, now growing at double-digit rates, will exceed $15 billion by 2015.”5
However, many of the most interesting examples of this third wave are coming from the edge: startups in Silicon Valley; labs at the Massachusetts Institute of Technology (MIT). These players are exploring powerful new ways to help link data and identity – and in the process are redefining how companies can know and even influence who they sell to.
The power of social data
Privacy concerns notwithstanding, these trends can be an extraordinary boon to consumers. The last decade saw the Web awash in a rapidly rising tide of information, leaving most people helpless to filter through mountains of unstructured data. Companies offered more products, advertisers pushed more messages and it simply became too difficult to manage. With social data, companies were given a powerful means to help cut through the clutter. Just as Netflix uses its Cinematch engine to find movies you might want to see, companies can personalize and tailor what they put in front of their customers – by using what they reveal about themselves online.
Partially enabling these trends is Facebook Connect, a technology that allows Facebook users to take their social networks with them around the Web. With Connect, outside developers can leverage your social data to build services and products around your social graph. Sony, for example, plans to use it to create personalized video games on the PlayStation 3 console. “We’ve pretty much opened up the entire Facebook API to our game developers,” said Eric Lempel, Sony Computer Entertainment’s vice president of Network Operations. “They’re able to pull any piece of information from Facebook, as well as push information out to Facebook.”6 With these kinds of flows, the next generation of games could very well have pictures of your friends or your tastes and interests built right in. Lempel envisions a future where “[a] music video game like Rock Band might know what music you’re a fan of on Facebook and customize your track list based on that.”
Professor Pentland demonstrated software that he described as 95 percent accurate at determining what company you worked for, simply by analyzing your interactions and patters of movement.
This kind of hyper-personalization gets more powerful as you start exploring the third wave. Companies like Media6Degrees, for example, are pioneering new forms of “social targeting” to help identify key influencers and focus on them for advertising campaigns. By honing in on individuals who are most likely to be receptive to messages from a brand—for instance, friends of existing customers—the company can deliver results that often outperform traditional demographic targeting. Recently, they raised an additional $20 million in funding and estimate that by tailoring ads based on your social graph they are two to 10 times more likely to get clicks.7
An important point about this third wave, however, is that it is hardly limited to data from social networks. In fact, some of the most compelling opportunities in social analytics lie far outside our relationships on Facebook.com. Instead, says Auren Hoffman, the CEO of Rapleaf, a leading data-mining company, “the next big data revolution will be around mobile. If you think about mobile devices as sensors, our phones know more about us than our partners do.”8 In fact, they are one of the most common sensors in existence: there are already more than five billion mobile phones in use worldwide, with another billion on the way in 2012.9
More importantly, in the U.S. alone, mobiles create around 600 billion geospatially tagged transactions per day.10 These data, properly harnessed, have the potential to provide a dynamic view of human behavior and activity, and dramatically augment our understanding of what takes place in the physical world.
There is certainly still a place for snapshot-style data, like surveys or market research, but when it comes to deeply understanding customers, the edge will likely go to those who do it in real-time.
One of the people doing this kind of analysis is Professor Alex (Sandy) Pentland of MIT. His work—which he calls Reality Mining—aims to understand and predict human behavior by combining real-time sensing of millions of mobile phones with advanced mathematical models. So far, he can predict things like what iPhone applications you are likely to buy and even describe a city’s cohesion, attitudes and behaviors by studying mobile location trails.11 At a recent technology conference, he demonstrated software that he described as 95 percent accurate at determining what company you worked for, simply by analyzing your interactions and patterns of movement.12
Others are starting to expand on Pentland’s work, and their findings have profound implications for both consumers and data-driven firms. A company called Sense Networks, for example, was founded by a former student of his and has developed a proprietary methodology to help extrapolate a person’s lifestyle—including their estimated age, their probability of being a business traveler, how wealthy they are, and where they might go next—all from their location trails. The app, called MacroSense, can do this even if you haven’t told it anything explicit about yourself. Instead, it is based on the notion that people with similar movements tend to have similar interests and demographics. By comparing where you go against billions of data points on the movements of others, the company can segment out “social tribes” and offer highly targeted opportunities.13
In general, the power of geospatial analysis is increasingly apparent, even outside of the tech world. The U.S. Court of Appeals for the District of Columbia, for instance, recently wrote: “A person who knows all of another’s travels can deduce whether he is a weekly churchgoer, a heavy drinker, a regular at the gym, an unfaithful husband, an outpatient receiving medical treatment, an associate of particular individuals or political groups—and not just one such fact about a person, but all such facts.”14
In this sense, perhaps the most important conclusion from this and the other examples in this section is that the universe of data we can draw upon to understand our customers is almost limitless. Social data needn’t be from social networks, and social analytics is much bigger than just listening to chatter on Facebook. To lead beyond today, companies will likely have to develop a deep and real-time understanding of their customers through a variety of digital trails.
Unlocking analytics in your organization
How can you start to develop these insights within your own organization? How can you finely tune your offerings or ads based on the data your customers create? A first and overarching step is to realize that “it’s not just about technology,” says Marc Benioff, chairman and CEO of Salesforce.com. Rather, “it’s about a fundamental shift into a new age of leadership with a new type of executives who behave and operate in new ways.”
This means no bolt-on solutions; no one-time customer surveys. Instead, you have to connect that technology, and data analytics as a whole, very deeply into your operating model to begin to see real results from it.
It doesn’t take all the data in the world, however. With so much data available, from so many different sources, it may actually be possible to build a 360 degree view of the customer; but for most companies, 15 degrees will do. The real challenge is making sure that you have the right 15 degrees.
At the same time, in today’s hypercompetitive, fast-moving environment, what you know about your customers is constantly diminishing in value. As the half-life of information shrinks, it is critically important not only to find the right data, but to constantly refresh what data you have. This means designing active flows of information, rather than collecting passive stocks. There is certainly still a place for snapshot-style data in this environment, like surveys or market research, but when it comes to deeply understanding customers, the edge will likely go to those who do it in real time.
The next and most difficult step relates to Benioff’s point above: building an engine to run your business on these flows. It isn’t enough to simply collect and analyze data. After all, data are only as valuable as how they are applied. Instead, companies will need to figure out not just how to extract information about their customers, but how to structure their organizations to act on it in real time; how to customize offerings on the fly; how to respond quickly to changes in consumer behavior; how to intelligently manage privacy and risk; how to segment, price and market effectively, as new information comes in. In short, the way Netflix operates—as a dynamic product, different for everyone who uses it—may foreshadow how leading firms behave down the line.
Reaching this point isn’t easy, but for those that do, the rewards can be significant. When Netflix uses your data, for instance, it isn’t just tailoring what movies you see; it is shaping behavior writ large. By serving up titles you might like but wouldn’t think to search for, the company is actively pushing you away from movies that cost it the most to license. Instead, renters find themselves watching fewer new releases and gradually expanding their tastes.15 In this sense, Netflix’s much lauded effort to cater to the individual can also add directly to its bottom line. As companies improve at making sense of behavior, the ability to shape it like this may emerge as a significant, if not key differentiator. What will set firms apart, however, is how well they are able to leverage the power of data in the second and third waves. With it, they can understand things like relationships and influence that are deeply important when trying to give consumers a nudge.
However, this doesn’t mean companies should marginalize the first wave or skip over the second in order to get to the third. Far from it. Instead, we think each wave reinforces and is made more powerful by the last.
Companies can achieve better results by striking the right balance among the three. And like all things data, even this will need a tailored fit. DR
Doug Palmer is a principal with Deloitte Consulting LLP.
Vikram Mahidhar is deputy leader of Innovation, Deloitte Services LP.
Dan Elbert is a consultant with Deloitte Consulting LLP.
2. Ian Ayres, Super Crunchers: Why Thinking-By-Numbers is the New Way To Be Smart, Bantam (2007)
4. Adam Ostrow, “Inside Gatorade’s Social Media Command Center,” Mashable, June 15, 2010, http://mashable.com/2010/06/15/gatoradesocial- media-mission-control/.
8. Conversation w/Auren Hoffman on Nov 9, 2010
9. “Over 5 Billion Mobile Phone Connections Worldwide.” BBC.co.uk. BBC, 9 July 2010. Web. <http://www.bbc.co.uk/news/10569081>.
10. Jonas, Jeff. “Your Movements Speak for Themselves: Space-Time Travel Data Is Analytic Super-Food!” Web log post. Jeff Jonas, 16 Aug. 2009. Web. <http://jeffjonas.typepad.com/jeff_jonas/2009/08/your-movements-speak-for-themselves-spacetime-travel-data-is-analytic-superfood.html>.
11. Kate Greene, “TR10: Reality Mining: Sandy Pentland Is Using Data Gathered by Cell Phones to Learn about Human Behavior,” Technology Review, March/April 2008, <http://www.technologyreview.com/communications/20247/page1/>.
12. Conversation with Alex (Sandy) Pentland on Nov 17, 2009 at MIT’s Media Lab