Let’s start with quantitative analysis like benefit-cost analysis (BCA) – there is a lot of experience behind this quantitative approach and is supported, mandated even, by central funding agencies, particularly Treasury departments.
So, let me start with BCA. This method is only as good as the data behind the analysis, which usually depends on demand forecast estimates in particular.
The quality of a BCA rests on the assumptions underpinning the forecasts and these are often not transparent, challenged or tested. The consultant or sub-consultant is relied on too often in my experience to produce a result without question – a ‘black box’ approach.
Many benefits of a transport investment considered in a business case are not readily quantifiable or monetised, so are essentially ignored.
There is not enough, or any data, or it is not cost-effective to collect data or do the analysis.
This is true of active transport and to a large extent public transport and transport operations, such as traffic management.
Other areas, take emissions as an example, there is no accepted ‘market price’ or no agreement on a price to quantify the benefits.
A BCA results in the focus on a single number such as the benefit-cost ratio (benefits/costs) or the net present value and this can be a problem if you cant value major benefits or the assumptions behind the forecast are not been reviewed and agreed.
So, BCA is not as robust as often made out to be.
Why emphasise qualitative assessment you ask?
There has been a lot of biased criticism of multi-criteria analysis (MCA).
MCA is not as problematic as it’s is often made out to be in my view. In fact, it is more robust by being able to take into the analysis all the potential benefits, whether they can be monetised, quantified or not. It does not have to be metric based but can be qualitatively evaluated using an agreed scale, such as 1 to 5 or -5 to + 5. The means of assessing should be done using a range of key stakeholders and be open and transparent.
A BCA should be embedded in the MCA, thus complementing the BCA and amplifying it. Properly conducted a MCA is robust process and has been part of UK government policy for decades, the latest edition (2018) applies to all proposals that concern public spending – must be time we caught up?
The Australian Transport Assessment and Planning (ATAP) guidelines and latest Building Queensland Guidelines include an Appraisal Summary Table (AST) to provide a summary of the analyses.
The increasing emphasis on various forms of risk analysis, including sustainability, social impact, environment, climate change etc also supports an increased use of qualitative assessments.
So in summary, qualitative assessments, such as MCA and AST should receive greater emphasis in the ATAP guidelines, and greater support by transport agencies and funders.
What do you think? Interested in your comments.
References:
Australian Transport Assessment and Planning. 2018. F3 Options Generation and Assessment
Building Queensland. 2020. Stage 3: Detailed Business Case. Guide.
UK HM Treasury. 2018. The Green Book: appraisal and evaluation in central government