Listening to today’s leaders, advisers and commentators talk about the parlous state of the civil service, and the faltering progress of civil service reform you could be forgiven for thinking that the history of such reform in the UK must be one of unalloyed failure.
But the reality is that the UK is a frenetic and creative public service reform innovator whose efforts are admired and copied overseas long after they have been wound up or condemned in the UK.
Working at the Institute for Government in London in 2014 Nehal Davidson and I had the luxury of six months to research some the most successful reforms in the UK with the aim of distilling the factors that explained why some reforms went well, and why other didn't. Our full report is available in full on the Institute for Government's excellent website.
In our initial interviews with past and present officials, non-executive directors and ministers a small of number of reforms were consistently named as having a lasting impact on the capability of the civil service:
the Next Steps report and the ‘agencification’ of the Civil Service (1987-98);
Bringing In and Bringing On Talent and subsequent efforts to improve diversity, talent management and career development in the Civil Service (1999-2002);
Introduction of Public Service Agreements and the Prime Minister’s Delivery Unit (1998-2010); and,
Capability Reviews (2005-12).
These reforms are all seen as a successful, albeit to differing degrees, because they introduced beneficial changes that survived the rise and fall of the leaders, teams, structures and programmes that constituted the reform itself. The accumulation of these reforms has fundamentally, if sometimes unintentionally, changed the Civil Service.
First, they changed how politicians and civil servants saw their world, roles and the effectiveness of the levers of government.
Second, they taught them new ways to work, which survived long after a specific reform was wound up – ‘transformative routines’ as the rather ugly academic language of capability describes them.
Sometimes the four reforms were designed to consciously build on previous reforms, but rarely did they contribute towards any greater design or direction pursued by the most senior leaders of the Civil Service.
Most reform interventions have a shelf life and need to be reinvented – something that is rarely done well. We have seen time and again how incoming leaders, anxious to build their own reputation, grab the opportunity to invent something new rather than building on and improving what came before. As a result opportunities to accelerate, refresh and embed these reforms were routinely missed.
Even in those rare circumstances when attempts were made to rethink, refresh and develop the reform, efforts were often thwarted by difficulties in managing crucial transitions such as a change in leadership at the official or political level. Consequently even the legacy of these more successful reforms represents an underachievement.
This underachievement of swathes of reform over time (as opposed to individual reforms at a particular moment) seems to be the result of three systemic issues :
Leadership is too personalised and fails to survive crucial transitions.
The role of senior civil servants as ‘stewards of the Civil Service’ is underdeveloped and somewhat contested.
There is an absence of a corporate leadership team to engage today’s and tomorrow’s leaders in developing a shared narrative of what the future Civil Service needs to look like and what is required to get there.
Taken together these issues describe a historic failing of institutional and corporate leadership.
Case Study 1: the Next Steps report and the ‘agencification’ of the Civil Service 1987-98
Published in 1998 the Next Steps report aimed to change the ‘bone structure’ of Whitehall. The initial reform posed a serious challenge to the control wielded by the Treasury, senior politicians and officials. It emerged from a successful attempt to refresh and reinvent existing reforms (Rayner’s ‘lasting reforms’ and the ‘scrutinies’) and was developed through an extensive phase of engagement and analysis. It began ambitiously, hiving off between 75 and 95% of the Civil Service to arm’s-length agencies.
The energy, drive and purposefully disruptive manner of Peter Kemp, the first head of the Next Steps unit,was crucial to getting the Next Steps machine up and running. He initially relied on Margaret Thatcher and Sir Robin Butler, then Cabinet Secretary, to calm tensions with recalcitrant politicians and permanent secretaries.
A few years into the reform opposition was muted and it was no longer seen as insurgent or controversial. An energetic central team developed clear processes to underpin agency creation and undertook extensive engagement.
By 1992 The new Prime Minister’s was persuaded by Richard Mottram, the new project manager ,that Next Steps could reinforce his priorities. Agencies were created at a faster rate, in more complex areas, by an expanded and experienced central team.
But the discipline and sense of purpose gradually decreased over time as agency creation became mechanistic. There were no substantial attempts to refresh the model or ensure it was effectively embedded. Instead, in 1997, the programme was officially closed.
The Next Steps report had far-reaching consequences. It introduced some ‘hard’ changes in the operating model of the Civil Service. Greater freedom from the centre for the agencies contributed to a greater focus on customers and led to tangible improvements in frontline public services. It opened up recruitment to a new generation of senior managers from outside the civil service.
Next Steps instilled a clearer idea of what officials were supposed to be delivering. This encouraged people to look closely at a department and ask what it was there for, laying the foundations for the Public Service Agreements (PSA) agenda, which identified objectives for each department from 1998.
Case Study 2: The Prime Minister’s Delivery Unit and Public Service Agreements 1998-2010
The introduction of Public Service Agreements (PSAs) was unexpected and unplanned – proposed by the Chancellor’s special adviser just days before the 1998 spending review was announced.
The regime was electrified by the Prime Minister’s desire to grab hold of the public services agenda after the 2001 election when he gave Michael Barber his ‘instruction to deliver’. He wanted to change the course of delivery in four departments to meet 17 ‘priority’ PSA targets. The strong operating model drew on a well-resourced, diverse team and a collaborative style of working. Prime ministerial support was integral to the operating model, governance and incentives. Barber was very careful to create a good fit between PMDU and Treasury PSAs.
In 2007 during the long drawn out transition from Brown to Blair, 2007 the treasury undertook a major overhaul of the system government’s performance management framework shifted the focus of PSAs to 30 comprehensive high-level outcomes that cut across departmental boundaries.
There were far fewer targets. New governance arrangements at the ministerial and official level were established. Now integrated into HMT, PMDU took a much more collaborative approach to developing PSAs and the delivery agreements. The cross cutting, shared nature of PSAs challenged the federal nature of the Civil Service, inevitably diluting the accountability felt by secretaries of state.
The experiment was brought to a premature end by the Coalition Government in 2010. That many inside and outside government saw the abolition of PSAs and their associated machinery as a mistake is perhaps now acknowledged by the recent re-emergence of much of the approach and tools of PMDU in the guise of the revamped Implementation Unit. The PSA machinery provided a ‘guiding star’ to the policy direction of the whole government.
There are compelling examples of improvements in performance on the Prime Minister’s top priorities. More fundamentally, the approach is seen to have made civil servants and ministers feel directly accountable for delivery.
How did our successful reforms defy the gruesome history of the more numerous failed or abandoned reforms?
The single most important finding of our research is that success depends on awareness, understanding and insight. In each of the reforms we explored, leaders understood the context and environment in which they were operating, set the ambition accordingly and had the right leadership and reform design in place to drive desired changes. For example, a reform that fundamentally challenges the federal structure of the Civil Service will need to be designed in a completely different way to a reform that aims to introduce some beneficial changes, but only in a selected number of departments or areas.
But however well the early stage of reform is handled, the continuation of a reform is highly dependent on its ability to weather significant transitions such as a change in leadership. A strong, competent central unit can provide stability during these periods, but there are critical moments when opportunities need to be seized in order to refresh, reinvent and develop the reform.
Our report identified 10 factors that lift or drag down a reform at different stages of its lifecycle. Five speak particularly to the issue of the leadership of reform across the civil service:
Personalised leadership. A committed senior leader, who is seen to really own the reform idea, invests personal time and sticks with it, will send a signal that reform is important enough for officials to respond to. Leaders also need to have the leverage and permission to drive reform through. Although seniority can help with this, the leader must already have established their credibility, connections and ways of working within Whitehall.
Building a lasting coalition of leaders around reform. The personal authority and credibility of a leader is critical to a reform ‘taking off’ early on, but the failure to broaden ownership to a wider group of civil service and political leaders at later stages can sow the seeds of decline. Too often these leaders fail to engage a wider cadre of leaders in developing and challenging the reform. Leaders and their teams must actively develop a collective leadership coalition around the reform to prevent it from limping on, tailing off or being discarded.
The right degree of political support. Being attuned to the concerns, interests and priorities of politicians is critical to making a sound judgement about the extent of political engagement required to make progress. It is important to remain aware of how political involvement can also hinder efforts; it may therefore be necessary to ‘disconnect’ the reform from particular politicians or parties.
The support, or at least permission, of the Treasury. The Treasury can electrify, undermine or suffocate any reform; many see it as the missing leader of civil service reform. Active involvement may not be necessary, but the Treasury certainly has to ‘allow’ reform in order for it to happen. The time required to engage, persuade and reassure the Treasury should not be underestimated.
Ambitious while connecting with departmental priorities. However ambitious and far-reaching the reform is, leaders must test and improve the design so that it chimes with the prevailing interests, concerns and priorities of officials rather than challenging them directly. This is not a ‘nice to have’, but is critical to designing the right operating model, governance and incentives to drive desired changes.
These five factors are right at the heart of the unresolved problem: how to creating lasting collective political and official leadership of the civil service? This crucial issue is still missing from today’s reform agenda. Any minister or official wishing to lead successful reform that survives their departure will need to address and overcome this problem.
This blog was originally published in Public Finance Magazine in 2014