Professor Bent Flyvbjerg in his research identified two main causes of misinformation in policy and management: strategic misrepresentation (or lying!) and optimism bias (appraisal optimism).
Strategic misrepresentation is the planned, systematic distortion or misstatement of fact in response to incentives in the budget process.
Optimism bias is the demonstrated systematic tendency for people to be overly optimistic about the outcome of planned actions. This includes over-estimating the likelihood of positive events and under-estimating the likelihood of negative events.
The underlying drive for the initial study (Flyvberg 2003) was the realisation that despite the large amounts of funds spent on infrastructure projects, there was little empirical evidence to demonstrate how well these projects performed in terms of actual costs, benefits and risks.
The study analysed a sample of 258 projects, in 20 nations, executed over a period of 70 years (1927–1998) with a total combined value of approximately US$90 billion (in 1995 prices).
The study’s main findings were:
- Cost escalation is the norm rather than the exception, with almost nine out of 10 projects exhibiting some level of cost overruns;
- Actual costs were on average 28% higher than forecast costs;
- The error of underestimating was statistically more common than the error of overestimating (which means that costs were underestimated more often than overestimated);
- Costs that have been underestimated were wrong by a substantially larger margin than costs that have been overestimated;
- Cost escalation has not decreased over the past 70 years (which means that we have not been seen to be doing a better job at estimation in recent years compared with the former years).
Cost estimates used in decision-making for transport infrastructure development are highly, systematically and significantly misleading.
Professor Bent Flyvbjerg
Flyvbjerg and his associates have developed methods to curb misinformation focused on improved accountability and reference class forecasting.
Reference class forecasting predicts the outcome of a planned action based on actual outcomes in a reference class of similar actions to that being forecast.
Kahneman and Tversky (1977) found that human judgment is generally optimistic due to overconfidence and insufficient consideration of distributional information about outcomes.
Therefore, people tend to underestimate the costs, completion times, and risks of planned actions, whereas they tend to overestimate the benefits of those same actions.
Using distributional information from previous ventures similar to the one being forecast is called taking an ‘outside view’.
Reference class forecasting is a method for taking an outside view on planned actions. Reference class forecasting for a specific project involves the following three steps:
- Identify a reference class of past, similar projects
- Establish a probability distribution for the selected reference class for the parameter that is being forecast
- Compare the specific project with the reference class distribution, in order to establish the most likely outcome for the specific project.
Further Reading
Flyvberg, B (2003) How common and how large are cost overruns in transport infrastructure projects? Transport Reviews, 23(1), p71–88.
Flyvbjerg, B. and COWI (2004) Procedures for Dealing with Optimism Bias in Transport Planning: Guidance Document (London: UK Department for Transport).
Kahneman, D. and Tversky, A. (1977) Intuitive Prediction: Biases and Corrective Procedures.