Large public transport projects require patronage demand estimates for planning and investment purposes. While patronage forecasting has been in use for over 50 years, providing the basis for determining the best option to a transport problem and guide investment decisions, they are more often than not, wrong!
Forecasts involve large data collection and analysis exercises using sophisticated strategic transport models to produce forecasts and estimates. Understanding and making judgements on critical assumptions in these models is complex and there are only a very limited number of experienced public transport forecasting professionals.
It should be remembered that all models are estimates, and all estimates are inherently unreliable, some are just optimistic, others are deliberate exaggeration.
Professor Bent Flyvbjerg [1] identified two main causes of misinformation in policy and management: strategic misrepresentation (lying) and optimism bias (appraisal optimism).
Strategic misrepresentation is the planned, systematic distortion or misstatement of fact in response to incentives in the funding and financing process.
Optimism bias or appraisal optimism is the demonstrated systematic tendency for people to be overly optimistic about the outcome of planned actions. This includes over-estimating the likelihood of benefits and under-estimating the likelihood of negative events and costs.
A recent global study conducted by Flyvbjerg [2] of more than 2,000 infrastructure projects concluded that cost-benefit analyses for large projects tend to overestimate cost-benefit ratios by between 50% and 200%!
Motivated Reasoning
There are a number of examples where estimates have been motivated by commercial interests. The winner of the bid for a private sector franchise will usually be the lowest cost to government, hence the incentive for the highest patronage forecast.
Market failures such as the initial public transport franchises in Melbourne, where private sector operators optimistically overestimated the potential patronage growth and failed, in one case walking away from the contract and sacrificing their multi-million dollar bond.
Private sector bidders in order to win contracts push assumptions behind estimates and models to the limit. This is illustrated by the number of recent examples of high profile failures of toll road projects in Australia.
This has resulted in an increased risk of litigation for inaccurate forecasts where large private investments are at stake, further reducing the number of forecasting consultants willing to be involved.
And there are examples where decisions based on projections and estimates are motivated by political interests. Presenting an attractive public transport project in an election campaign wins seats!
The recent Grattan Institute report [3] criticises transport investment proposals and decisions of governments, in many cases without a published business case, or dubious benefit-cost results. For example:
Canberra’s light rail, now being built, is likely to provide no more benefits than bus rapid transit but cost more than twice as much.
“The business case for Canberra light rail, published in 2014, reported an estimated benefit cost ratio of 1.2. However, land use benefits and wider economic impacts, which are typically excluded from project evaluations by Infrastructure Australia because the risks of overestimating them are so high, account for almost three fifths of the projected benefits. If these land use benefits and wider economic impacts are excluded, the benefit- cost ratio is just 0.5 – well below the level needed to deliver a net benefit to the community.
The Melbourne Metro business case stated the benefit-cost ratio increased from 1.1 to 1.5 with the inclusion of wider economic benefits.
What can be done?
The challenges in forecasting public transport patronage are many, including insufficient current data on travel patterns, the response to new services and fare policy changes, understanding travel behaviour, and investing in the ongoing development of strategic transport models.
Professionals typically underestimate the cost and overestimate the benefits of transport projects.
To improve patronage projections requires many of the challenges to be addressed, at least in part:
- Independent collection of data on demand characteristics and travel patterns and forecasts of demand from population, economic and land use projections
- Establish specific travel behaviour and elasticities of demand for new services and fare changes
- Provide independent forecasting and benchmarks to reduce potential for strategic misrepresentation and optimism bias.
Project business cases typically underestimate costs and overestimate benefits. How can this be improved?
References:
[1] Professor Bent Flyvbjerg from Oxford University is the most cited scholar in the world in megaproject management. The “iron law of megaprojects,” he wrote in a 2014 paper in Project Management Journal, is that they are “over budget, over time, over and over again.”
[2] Flyvbjerg, B. and Sunstein, C. (2015) The Principle of the Malevolent Hiding Hand; or, the Planning Fallacy Writ Large, Social Research, Forthcoming.
[3] Marion Terrill. 2016. Roads to riches: better transport investment. Grattan Institute.