We need experimental innovation policies to transition to a greener economy that works for people and planet

Why increasing R&D budgets alone won’t solve the climate crisis and how experimentation offers innovation policymakers a life line

The COVID-19 pandemic accelerated the rate of change for many sectors, disrupting as well as transforming how we live and work. The need to rethink our approach to economic growth has never been more pertinent, with the climate crisis high on the global agenda and the upcoming United Nations Climate Change Conference (COP26). As experts continue to tell us to brace ourselves for more frequent and intense extreme events, we are all becoming acutely aware of the finite resources of the planet. But if we are to learn anything from the world’s response to COVID-19, it is that innovation offers us a chance to accelerate rapid transformation towards greener economies. It is possible to ensure growth will not cost the world - a part of Nesta’s core mission - but traditional ways of rolling out innovation and growth policies will have to change. For years, the Innovation Growth Lab at Nesta have argued that innovation policies are not very innovative - millions get poured into policies where we have little evidence of what works and new ideas rarely get tried or tested. With the budget for R&D in many OECD countries having significantly increased to support mission-driven innovation agendas, our call for more experimental innovation policy that is evidence based is paramount to ensure effective transitions towards an inclusive green economy. 

"An inclusive green economy is one that improves human well-being and builds social equity while reducing environmental risks and scarcities."

United Nations Environment Programme (UNEP)

What is experimental policy?

Policymakers often face challenges without clear solutions and with many options to consider. Under the pressure to act fast, it is usually the existing policy or a single idea that is taken forward. Billions are then spent every year on innovation and growth policies, with little credible evidence of what works and what doesn’t. Out of all the evaluations of innovation schemes alone, only 3.7% were shown to have established a credible counterfactual to provide strong evidence of causality with 0.4% showing an impact. 

Experimental policies allow us to start small, trial different designs systematically and learn what works before we scale up solutions. This not only derisks new programmes but also allows for continuous improvement that help to save public money and ensure real-world impact. There are a range of ways to experiment, from rapid-fire trials through to randomised controlled trials (RCTs). Policymakers can draw on different approaches, depending on whether they are trying to better understand a problem, optimise existing programmes, or develop entirely new solutions. However, to be experimental it is not sufficient to pilot new programmes. Trying new things is important, but real experimentation requires putting in place the systems to learn whether policies are working or not and to use this information to inform future decision making.

1. Adaptation

Building a greener economy will not happen all at once. Change is likely to take place through a series of tweaks and adaptations that secure systemic change over time, edging us closer towards sustainable growth and inclusive innovations - because let us not forget who the climate crisis affects most. The ways we adapt must be informed by robust evidence and experimental policies that build bodies of evidence of what works at a local level before the diffusion of innovation to other regions - there will be no one-size-fits-all model for the world. 

What does this look like in practice? New technologies will play a prominent role but society will need to adapt to these changes. Nudging behavioral change requires an experimental approach to ensure that the intended impact of a policy is in fact taking place. For example, in the Netherlands around 1,100 companies with relatively high energy consumption agreed with the government to achieve an energy saving of 30% between 2005-2020. However, in 2015 it became clear that the feedback reports sent periodically to help these same companies meet their target were not being downloaded. An experiment was launched to test whether an alternative format delivering the same report would increase the number of downloads. This experiment not only showed that an alternative message, alongside sector comparison data increased the number of report downloads but also that it led to findings being discussed with colleagues and management more frequently. This relatively small tweak made a significant difference to ensuring the successful delivery of this policy. The more robust tests that are conducted at a local level, the greater the evidence base that may inspire different ways to tackle the global challenge of achieving net zero.

2. New ideas

Building a greener economy will also require policymakers to embrace new ideas, but traditional policymaking is usually risk averse - often for good reason. Experimental policymaking again offers us a way forward. For innovation agencies with finite resources, experiments help to explore the unknown by derisking potential solutions that have little evidence but high potential impact. By prototyping, running pilots and randomised controlled trials, kernels of ideas may be tested at various stages to build evidence with the degree of proof needed to validate assumptions on whether a particular programme or delivery methods are headed in the right direction. Experimental policymaking is a culture we urgently need to cultivate within our institutions. 

One particular sector of the economy that will benefit highly from new ideas that have been tested will be small to medium-sized enterprises (SMEs), who are responsible for a large proportion of carbon production although often overlooked. To ensure SMEs adopt cleaner technologies and best practices, new regulatory or tax incentives will be needed to ensure fast and optimal transition. However, getting SMEs to comply with environmental regulation will also require the optimisation of programmes to support SMEs struggling to make the necessary changes. Randomised controlled trials testing behavioral nudges as well as different programme delivery can help ensure businesses are more likely to engage with carbon reducing incentives that work.  

3. Focus on scaling what works

With the climate crisis looming, we cannot afford to pour money into policies that may or may not work. All facets of the economy - every company, bank, insurer and investor - will need to change their model to ensure decarbonisation happens at scale. At the same time, we can also not afford to further divide our societies with innovations that do not serve us all, making inclusive innovation policies paramount for a more sustainable future. For innovation policymakers, this multilayered challenge to ensure high growth and increased productivity while securing an inclusive greener economy can be a massive challenge - something which is continuously evolving as it did during COVID-19. Scaling is no longer about placing bets on one solution but rather testing a wide range of promising ideas to only further invest in what meets the standards of evidence required.

A good example of experimental design can be seen with the UK’s Department for Business, Energy and Industrial Strategy (BEIS) who launched the Business Basics Programme to support SMEs to adopt proven management practices and technologies. Rather than picking one single design, the £9million programme was designed to test a range of innovative ways to approach the focus, design and delivery of interventions. Through the “experimentation fund”, delivered in partnership with Innovate UK and IGL, BEIS supported a total of 31 policy experiments and pilots. As they support SMEs these projects are generating a wide range of evidence, with positive stories about success in achieving impacts but also many cases where an intervention is found not to work in its current format - saving a lot of money compared to a scenario where one of these had been the single approach delivered at larger scale. Experimental policies help ensure that only what works gets further investment of finite public and private sector money.

4. More regular feedback loops

At this point you may be wondering, when does experimentation end? The more feedback loops our innovation policies have, the greater the levels of certainty that policies and programmes will lead to the desired impact. This means using experimentation until we have the degree of confidence we need. Experiments can be used in lots of different ways so depending on the maturity of a policy we may use a different experimental method for varying degrees of time - at the start we may be testing a lot of assumptions while more mature policies may need to be optimized during the delivery phase. Experimental policies build a culture where evidence informs decisions rather than hunches. The more comfortable policymakers can get with exploring the unknown and using experiments to test assumptions about how best to transition our economy, so that it works for people and planet, the greater our chances of achieving an inclusive sustainable future.

A final word from us...

At the Innovation Growth Lab we are strong advocates for the use of randomised controlled trials (RCTs) because beyond measuring effect (i.e. for impact evaluations) RCTs can also be used to test underlying drivers of a problem, explore various mechanisms at once to inform the final design of a programme or compare different policy implementations to optimise the final delivery. Experimental policies are not the silver bullet to transition us to greener economies but they are a strong tool to centre inclusive innovation by ensuring local, robust evidence informs shared solutions to a global challenge. We will be sharing new ideas on how experimentation can support transformative innovation policies in the coming months and at the Transformative Innovation Policy conference in January 2022. To keep in touch with us and with our work sign up to the IGL newsletter or get in touch with us directly at [email protected]. 

This blog was written with the support of my colleagues Alex Glennie, Senior Policy Manager and James Phipps, Head of Economic Analysis and Policy Development at the Innovation Growth Lab.