I promised to review the Infrastructure Risk Group report on Managing Cost Risk & Uncertainty In Infrastructure Projects, a report which was launched last month at the ICE with the sponsorship of the IRM.

The report is a result of work undertaken as part of the Infrastructure UK investigation of the high cost of infrastructure projects.  (IUK is part of the Treasury.)  One of the prime suspects in this investigation is the high level of financial contingency allocated to projects.  Whilst it might be  prudent to create these contingency funds, there is a suspicion that they always gets spent, whether strictly necessary or not.  You only have to see the current political wrangling over HS2 to understand the importance of the issue, as well as how unlikely it is to go away.  And it’s an obvious point that HS2 follows hot on the heels of the Olympics in this respect.

On this occasion I’m not an entirely disinterested observer.  A glance inside the cover shows that I am (or possibly was) a member of the IRG.  So there’s a lot in this report I like.

The big story is that one source of the large contingencies is optimism bias, the fiddle factor that HMT applies to projects to get from what optimistic engineers tell them to something that history says is realistic.  I believe optimism bias was invented by Gordon Brown out of frustration with the unrealistic costs – as shown by outturns – the Treasury was being presented with.  The numbers are big – up to 66% added on for ‘non-standard civil engineering projects’ – so if you could avoid spending it you save a lot of money to spend on other things.

The problems are:

• the costs are uncertain
• it’s hard to characterise this uncertainty; there’s no objective measure
• even if you could characterise it objectively, how much of a worst case do you provide for financially?
• because it’s subjective it’s easy for people to play games: to inflate the provision and to hold onto it when it’s no longer necessary.

The strength of the IRG report is that it confronts these matters honestly instead of pretending that all you need is better, or more,  risk assessment.  The first of its nine recommendations suggests that forecast  costs be presented as a range instead of a single number.  And the second proposes that allowances for risk be based on analysis and not just global optimism bias corrections.  This is important stuff and will shortly be incorporated in HMT’s Green Book guidance on project appraisal via supplementary guidance.  [By coincidence the supplementary guidance was published the day aftere this review was posted.  You can find it on gov.uk and I’ll add a comment when I’ve had a chance to see what’s actually been posted.  The prospective death of optimism bias seems to have been exaggerated!]

Some of the remaining recommendations focus on improving risk management (the doing what we said we’d do problem) and the management of financial contingency pots.  The others set out a continuing programme of work for the IRG which envisages it being ‘formally established as the guardians of leading UK practice in project risk and contingency management.’  I hope the IRG has the enthusiasm and clout to deliver on this as it has a significant contribution to make.

Returning to the report, I think it has two real strengths:

• A commitment to conceptual clarity: it clearly differentiates between risk analysis and financial management processes such as budgeting or contingency drawdown, between the use of one number to cover risk and a range or distribution.  To support this it has a glossary, not a comprehensive risk management glossary, but a list of terms which are used in a precise way with the report and which need  to be used in a precise way to convey the meaning.  I’ve mentioned before that this glossary distinguishes between the concept of ‘risk’ as the existence of uncertainty and ‘a risk’ as events to be listed.  All writers of risk standards and guidance should shrivel up and die of embarrassment at this point!
• Case studies: six companies (London Underground, Network Rail, Heathrow, Olympics, Crossrail, Highways Agency) discussed their risk processes with the IRG and allowed this to be published.  Perhaps the most interesting of these is the ODA, the Olympic Delivery Authority.  Their processes seemed very well-defined, and in fact I’ve discussed this before on this site.  I think they form a template which everyone should use.

Something the report misses is the opportunity to provide what you might call a unified view of risk analysis, whether it is done from risk lists, project experience or fiddle factors.  The diagrams seem to be on the verge of illustrating this, but this omission means that the risk listing approach is presented as the only way, particularly at the expense of reference class forecasting (the name which for some reason is used for risk estimates based on project experience).  This means the report is somewhat self-contradictory as reference class forecasting is quoted with approval elsewhere, including its widespread deployment in Network Rail.  Associated with this is a slightly mysterious omission of any description of the victim, optimism bias, which was actually quite a well-founded, if slightly outdated as well as inconvenient, risk analysis technique.

This Soviet-style airbrushing of history is quite unfortunate as the real problem with optimism bias was that everyone forgot the Green Book guidance that it should be updated and replaced by appropriate risk analysis as the project is developed.  We need to beware of new orthodoxies which, used without thought, create bad practice.  One day I’ll do a post on ‘unified risk analysis’  (part of the relativistic risk analysis project).

Will the IRG report do any good in practice?  For that to happen people will have to read the report and act on its recommendations.  The end product has to be

1. good project selection and definition
2. enough money to deliver these projects in the face of uncertainty
3. but none of it spent unnecessarily.

As and when the supplementary Green Book guidance comes out, it should be a great help.  But I think the important point is that the IRG should keep going, keep pushing.  If it keeps up its current willingness to question, explain, and act as guardian of data and tools, it has the potential to make real difference to the way we analyse risk and use the results to inform our spending decisions and manage cost.

So go forth and read!