A Random Search for Excellence
The Persistence Project
Many believe we can learn how to be great by studying greatness. But what is great performance? It turns out that most studies typically measure the wrong things and set the bar far too low. Consequently, researchers who think they’re looking at successful companies are usually studying the winners of a random walk. What does this mean for the soundness of some of the most popular and influential management studies? Simply this: You can’t trust them.
The following themes are excerpts from “A Random Search for Excellence." The complete article can be downloaded at the bottom of this page.
Randomness In, Randomness Out
Many managers have found the prescriptions in one or more of these studies helpful, perhaps even enormously so. And we’re sympathetic to the notion that if it works, don’t knock it. But we’ve come to the disturbing conclusion that every one of the studies that we’ve investigated in detail is subject to a fundamental, irremediable flaw that leaves us with no good scientific reason to have confidence in their findings.
It is this: success studies aren’t studying unexpectedly successful companies. Rather, they are, by an overwhelming majority, studying a collection of firms that have taken fortunate random walks. In other words, they are not studying great companies, they are studying lucky companies. Consequently, the inputs to every success study we can lay our hands on are the wrong inputs. This has material consequences for the confidence we can have in the advice offered, for no matter how rigorous the data collection, no matter how Aristotelian the logic, to deviate a bit from the old aphorism, “randomness in, randomness out.”
Finding the Right Measures of Success
When looking for outperforming companies, the first question is always how do you measure performance?
A common, but hardly unanimous, practice is to look at total returns to shareholders (TSR): Blueprint to a billion used sales, Built to Last used a composite measure of general esteem, The Breakthrough Company used primarily sales growth. Some studies wanted to examine a specific phenomenon (how to get to a billion in sales), while others had to cope with constraints on data availability (if you’re looking at private companies, you can’t use TSR).
We took the view that at first principles, the object of all of these studies, ours included, is to identify noteworthy management practices. This has some important implications, of which perhaps the most surprising is that TSR is arguably the worst measure to use.
Shareholder returns are a function of the capital market’s estimate of future performance. A good fraction of TSR tells the story of changing hopes for the future rather than delivering on past promises. Consequently, strong returns over time are often largely the result of consistent upside surprises that serve to ratchet up expectations, which is then made manifest in a rising stock price.
This is perhaps most evident in companies that deliver great performance but lackluster TSR: a company can deliver fabulous profitability and eye-popping growth and have a share price that goes nowhere because at some point in the past the markets “figured them out” and priced that performance into the shares.
Now, if one is interested in what behaviors surprise investors, TSR might be just the ticket. But if great management is what you want to understand, operating measures of performance are much better. Of the operating measures available, we like return on assets as an overall measure of profitability.
Setting the Standards of Excellence
There’s one more step before we can determine what constitutes statistically remarkable performance: we have to define a benchmark. For the purposes of our success study, we have defined two categories of exceptional performance, tentatively labeled “Miracle Workers” and “Long Runners.” The former deliver whatever number of 9 th decile years is statistically unlikely given their lifespans, while the latter deliver whatever number of years in the 6 th-8 th decile band that is similarly improbable. Our definition of excellence is motivated by a desire to contrast the behaviors of firms that do exceptionally well for a long period of time with those who do merely well (rather than exceptionally well) over long periods.
Finally, we have identified “Average Joes” – companies with a statistically unremarkable life span, performance level, and performance variability. Our intent, in the time honored tradition of the success study, is to compare and contrast the behaviors of these firms and thereby tease out the necessary and sufficient conditions for exceptional performance.
The Fable of the Fables
You shouldn’t necessarily dismiss the advice offered in success studies. Many authors are savvy observers of the business world. Their recommendations can be useful, but more in the manner of fables than evidence-based advice. And we use fables very differently from science.
For example, no one reads “The Tortoise and the Hare” and, faced with a chance to bet on such a race, chooses the tortoise. Rather, people take from this tale the idea that there is merit in perseverance while arrogance can lead to a downfall. Similarly, because the prescriptions of most success studies lack an empirical foundation, they should not be treated as how-to manuals, but as a source of inspiration and fuel for introspection. In short, their value is not what you read in them, but you read into them.
Download the full monograph below.