**The Flaw of Averages**by Sam Savage

A version of this article appeared in the November 2002 issue of *Harvard Business Review*.Consider the case of the statistician who drowns while fording a river that he calculates is, on average, three feet deep. If he were alive to tell the tale, he would expound on the “flaw of averages,” which states, simply, that

**plans based on assumptions about average conditions usually go wrong.** This basic but almost always unseen flaw shows up everywhere in business, distorting accounts, undermining forecasts, and dooming apparently well-considered projects to disappointing results.

Let’s say that a company I’ll call HealthCeuticals sells a perishable antibiotic. Although demand for the drug varies, for years the average monthly demand has been 5,000 units, so that’s the quantity the company currently stocks. One day, the boss appears. “Give me a forecast of demand for next year,” he says to his product manager. “I need it to estimate inventory cost for the budget.” The product manager responds, “Demand varies from month to month. Here, let me give you a distribution.” But the boss doesn’t want a “distribution.” “Give me a number!” he insists. “Well,” the manager says meekly, “the average demand is 5,000 units a month. So, if you need a single number, go with 5,000.”

The boss now proceeds to estimate inventory costs, which are calculated as follows: If monthly demand is less than the amount stocked, the firm incurs a spoilage cost of $50 per unsold unit. On the other hand, if the demand is greater than the amount stocked, the firm must air-freight the extra units at an increased cost of $150 each. These are the only two costs that depend on the accuracy of the forecast. The boss has developed a spreadsheet model to calculate the costs associated with any given demand and amount stocked. Since the average demand is 5,000 units, he plugs in 5,000. Since the company always stocks 5,000 units, the spreadsheet dutifully reports that for this average demand, the cost is zero: no spoilage or airfreight costs.

A bottom line based on average assumptions should be the average bottom line, right? It may miss fluctuations from month to month, but shouldn’t you at least get the correct average cost by plugging in average demand? No. It’s easy to see that the average cost can’t be zero by noting that when demand for HealthCeuticals’ antibiotic deviates either up or down from the average,

**the company incurs costs.****Show Me the Number**Executives’ desire to work with “a number,” to plug in an average figure, is legendary. But whenever an average is used to represent an uncertain quantity, it ends up distorting the results because it ignores the impact of the inevitable variations.

**Averages routinely gum up accounting, investments, sales, production planning, even weather forecasting.** Even the Generally Accepted Accounting Principles sanction the “flaw,” requiring that uncertainties such as bad debt be entered as single numbers. (To its credit, the SEC has proposed new rules that would begin to address this problem.)

In one celebrated, real-life case, relying on averages forced Orange County, California, into insolvency. In the summer of 1994, interest rates were low and were expected to remain so. Officials painted a rosy picture of the county’s financial portfolio based on this expected future behavior of interest rates. But had they explicitly considered the well-documented range of interest-rate uncertainties, instead of a single, average interest-rate scenario, they would have seen that there was a 5% chance of losing $1 billion or more—which is exactly what happened.

**The average hid the enormous riskiness of their investments.**More recently, a failure to appreciate the flaw led to $2 billion in property damage in North Dakota. In 1997, the U.S. Weather Service forecast that North Dakota’s rising Red River would crest at 49 feet. Officials in Grand Forks made flood management plans based on this single figure, which represented an average. In fact,

**the river crested above 50 feet, breaching the dikes, and unleashing a flood that forced 50,000 people from their homes.****Fixing the Flaw** Some executives are already attuned to

**the importance of acting on a range of relevant numbers—a distribution—rather than single values**, and they employ statisticians who estimate the distributions of everything from stock prices to electricity demand. Increasingly, companies are also turning to software-based cures for the flaw. Many programs now simulate uncertainty, generating thousands of possible values for a given scenario—in essence, replacing the low-resolution “snapshot” of a single average number with a detailed “movie.” The movie comprises a whole range of possible values and their likelihood of occurring—the frequency distribution.

The simplest and most popular tool, called the Monte Carlo simulation, was described by David Hertz in a 1964 HBR article and popularized in financial circles by sophisticated users like Merck CFO and executive vice president Judy Lewent. Today, spreadsheet-based Monte Carlo simulation software is widely available and is being used in fields as diverse as petroleum exploration, financial engineering, defense, banking, and retirement portfolio planning. Wells Fargo Bank, for instance, used a Monte Carlo simulation to predict the cost of offering customers a variable-rate CD, whose return would increase if interest rates rose. A previous estimate based on three years of 1990s interest-rate data had shown that the cost would be about 0.10% for a five-year CD. But the Monte Carlo simulation, which combined interest-rate data going back to 1965 with models of customer behavior, found that the bank’s cost could be eight times that amount. The alarming finding induced the bank to reconfigure its CD product to reduce the chance of unacceptable costs should interest rates rise.

Had the average-obsessed boss at HealthCeuticals used Monte Carlo simulation, he would have seen not only that the average inventory cost was not zero but that he shouldn’t have been stocking 5,000 units in the first place.

**For executives like him who are still fond of single values, it’s time for a shift in mind-set. Rather than “Give me a number for my report,” what every executive should be saying is “Give me a distribution for my simulation.”**