This is a huge gap in Whitehall but the system has gone so wrong few even realise the gap is there and those who do cannot do anything about it.

Over the weekend I read The Hollow Men II: Some reflections on Westminster and Whitehall dysfunction, an essay by Dominic Cummings. I thought it was excellent. Not least because it cites my hobby horse - the poor understanding and mis-management of complexity - as the principal cause of dysfunction in government.

The essay is long - 23,000 words. You should read it. Really. But just in case, this post is a bullet-point summary, with some quotes, that I thought might stand in. The titles match those in the essay...

Part I: ‘A combustible mixture of ignorance and power’

Our world is extremely unpredictable. Although good mechanisms for coping with complexity are emerging in markets and scientific fields, the same is not true in government…

All parties and the media are locked into a game that to outsiders is obviously broken – a set of implicit rules about the conduct of politics, and definitions of effective action, that tie them to behaviour that seems awful to the public, which is objectively failing, but from which they cannot free themselves. Because the system is stuck in a vicious circle – held in place by feedback loops between people, ideas, and institutions – whatever the outcome of the next election, the big problems will remain, No10 will continue to hurtle from crisis to crisis with no priorities and no understanding of how to get things done, the civil service will fail repeatedly and waste billions, the media will continue obsessing on the new rather than the important, and the public will continue to fume with rage.

This lack of capability causes serious problems…

There is therefore a mismatch between a) the growing reach of technology and the fragility of our civilisation, and b) the quality of elite decision makers and their institutions’ capacity to cope with these technologies and fragilities. Carl Sagan called this mismatch ‘a combustible mixture of ignorance and power’. If this mismatch persists, if we continue to pursue ‘traditional politics’ in the context of contemporary civilisation, it will sooner or later blow up in our faces.

We are not developing enough people with the skills to cope with complexity, and those we have are focused on the wrong things…

sucking a huge proportion of the cleverest and most expensively educated people in the world into high-frequency algorithmic trading (in which, for example, advanced physics is used to calculate relativistic effects that bring nanosecond trading advantages) is an obvious extreme mismatch between talent and priority.

Part II: Four Stories

This inability to cope with complexity is leading to bedlam in Whitehall, which Cummings illustrates with four (very entertaining) stories from his experience. In one he tells of his standard advice to new people arriving in his department…

assume that everything that can go wrong will go wrong and when you catch yourself thinking ‘someone MUST have done X or it would be crazy’, stop, because X will not be happening. Your only hope is to focus on a few priorities relentlessly and chase every day and every week.

In another he explains…

…what is really happening in Whitehall: most of everybody’s day is spent just battling entropy – it is not pursuing priorities and building valuable things.

Part III: Analysis

wrong people, bad education and training, dysfunctional institutions with no architecture for fixing errors

Mentality

Leaders in government are prone to self-deception about the extent to which they can influence complex systems…

Most of our politics is still conducted with the morality and the language of the simple primitive hunter-gatherer tribe: ‘which chief shall we shout for to solve our problems?’ …

…Such instincts, which evolved in relatively simple prehistoric environments involving relatively small numbers of known people and relatively simple problems (like a few dozen enemies a few miles away), cause disaster when the problem is something like ‘how to approach an astronomically complex system such as health provision for millions.’

Education and training

Leaders don’t have the right education or background…

MPs and officials have to make constant forecasts but have little idea about how to make them, how the statistics and computer models underlying these forecasts work, or how to judge the reliability of their own views.

students leave university for politics and the civil service with degrees that reward verbal fluency, some fragments of philosophy, little knowledge of maths or science, and confidence in a sort of arrogant bluffing combined with ignorance about how to get anything done. They think they are prepared to ‘run the country’ but many cannot run their own diaries.

The apex of the political system is full of people who have never managed employees on the scale of 10² people or budgets on the scale of 10⁶ pounds, yet their job is to reshape bureaucracies on scales of 10⁴ (DfE) – 10⁶ (NHS) employees and 10¹⁰-10¹¹ pounds.

Institutions and tools

The bureaucracy that exists in government is wholly un-suited to handling complexity…

They have big problems with defining goals, selecting and promoting people, misaligned incentives, misaligned timescales, a failure of ‘information aggregation’, and a lack of competition (in normal environments). These problems produce two symptoms: a) errors are not admitted and b) the fast adaptation needed to cope with complexity does not happen.

What sort of people are selected by parties to be MPs and (in the UK) form the pool from which ministers are chosen? MPs are seldom selected for their ability to devise policy, prioritise, manage complex organisations, or admit and fix errors. Elected representatives are often chosen from a subset of people who have very high opinions of themselves and who really enjoy social networking.

when someone with a startup mentality strays into the bureaucratic world, the bureaucracy reacts like an immune system to expel the intruder. This is one of the reasons why young talented people who want to get things done more than they want to get ahead … soon leave the civil service.

Timescales, planning horizons, and pace

Short-termism is an important problem. It might take several decades to make some changes effectively, yet the career demands of politicians and officials are typically less than 5 years. Further…

…nobody is incentivised to solve problems fast. Ministers acquire a reputation for ‘wisdom’ simply by saying about everything ‘sounds very risky let’s not do that’ or ‘let’s add another two years to the timetable’.

The failure of aggregation

There is bias in the aggregation and assessment of information…

High status people tend to dominate discussion and common information is over-discussed while information unique to an individual, especially a lower status individual, is routinely ignored.

The failure of core skills

Basic things go wrong because of poor skills…

Many of these bureaucracies cannot reliably do the simplest things. … Basic spreadsheet skills were so lacking that financial models and budgets could never be trusted and almost every figure released to the media or Parliament was wrong. Legal advice was unreliable and government lawyers are also given the wrong incentive (they are told to prioritise never going to court, which is stupid). Basic project management skills of the sort a world class engineering company routinely deploys are practically non-existent among senior officials.

Particularly, senior management is weak…

The fundamental reason for Whitehall’s failure is management, not a lack of bureaucrats or money.

Lack of internal criticism and external competition

There are poor mechanisms for identifying problems in advance and for genuinely learning from failure…

…officials would often refuse to have a proper look into the origins of a blunder. They would say that we could not look at all the documents on the grounds that ‘we must protect the convention that current ministers cannot look at the papers for previous governments’. This is very handy as under the cloak of ‘political impartiality’ officials prevent ministers getting to the bottom of complex long-term debacles. E.g. we were forbidden from seeing various documents about capital pre–2010 on the basis of ‘impartiality’ but when we insisted / tricked our way in, we found many cockups that Whitehall simply did not want revealed to anybody.

Barriers to entry to politics and the civil service mean that it is hard for those who would change the system, to become part of it…

Whitehall has such a tight grip on the MPs that it chokes off attempts to change the basic wiring of the system.

Conclusion

The essay concludes with stories of some recent events and an analysis of some of the characters involved…

The occupants of No10, like Tolstoy’s characters in War and Peace, are blown around by forces they do not comprehend as they gossip, intrigue, and babble to the media. The MPs and spin doctors steer their priorities according to the rapidly shifting sands of the pundits who they are all spinning, while the pundits shift (to some extent unconsciously) according to the polls. The outcome? Everybody rushes around in tailspins assembling circular firing squads while the real dynamics of opinion play out largely untouched by their conscious actions.

It ends with a trailer for a future post on What is to be done. Which I am looking forward to reading.