Building a community of experimental innovation agencies: insights from IGL’s work with TAFTIE

At the Innovation Growth Lab (IGL), we support innovation agencies - among other organisations -  to take a more experimental and data-driven approach to what they do. This involves helping them to build structured systems for learning through the application of experimental methods, and to use data and evidence in ways that will enable them to increase the impact of their activities. There are many benefits to becoming an experimental organisation, but it requires taking on new ways of thinking and working, and is not something that can be done by individuals in isolation. So IGL also seeks to build learning networks and peer support systems that can connect organisations taking this journey, allowing them to share and learn from each other while simultaneously developing their own skills and knowledge. 

Between 2019 and 2022, IGL and the TAFTIE network of European innovation agencies collaborated on a task force designed to help agencies from across the network to gain experience in experimental approaches through ‘learning by doing’. The objective for participating agencies was to develop their knowledge of how to design and run different types of experiment, and then to identify practical opportunities within their own organisations and programmes to set an experiment in motion.

At the kick-off meeting in Luxembourg in October 2019, none of us could have anticipated the huge upheaval to come. Covid-19 required a sharp pivot for everyone. Innovation agencies were occupied with rapidly rolling out new forms of support for innovators, and adapting to fully digital forms of interaction with their clients and partners. Our task force also had to change course, as we moved all of our plans online and adopted a more flexible agenda. Our original ambition for each agency to design and run a full experiment shifted to supporting them to develop their openness and capabilities to do so at a more favourable moment. Yet these unprecedented circumstances provided the perfect test bed to unearth the many push and pull factors that influence agencies’ journeys to becoming experimental. 

Reflecting on our work with TAFTIE, as well as other learning programmes we have been running in recent years, here are some key takeaways on how innovation agencies can get the experimental ball rolling, and then maintain momentum and build towards becoming experienced experimenters.

1. Embed the principles of experimentation 

At its core, experimentation involves trying new ideas, while putting systems in place to learn and provide a basis for decisions. It requires a willingness to take managed risks, and an openness to iterating and pivoting policies and programmes while they are still underway. This can be a big shift in approach for agencies that are used to rolling out funding calls or other support programmes and waiting until they are complete before evaluating them. In our experience, agencies are more successful in making this shift if they begin by getting really clear on why they want to experiment, and identifying how it will work in the context of their own organisational systems and processes. For example, the Netherlands Enterprise Agency (RVO) participated in the TAFTIE task force, and have subsequently joined IGL as a partner. The progress they have made so far has been accelerated by the fact that there is an organisational mandate and desire to embrace experimental methods and get stuck into designing and running trials to improve their programmes and ways of working. RVO have also recognised the importance of institutional culture in helping them to experiment, and have attached this work to a unit that is responsible for driving internal innovation.

2. Start small, and seize opportunities when they arise

The prospect of running large scale trials can seem overwhelming for agencies who are at the early stages of taking an experimental approach. The Austrian Research Promotion Agency (FFG) - lead agency of the TAFTIE task force and long time IGL partners - grew their organisational capabilities slowly, starting with relatively straightforward and easy to implement messaging trials that helped demonstrate the value of experimentation more widely across the organisation. Individuals within FFG leading this work have had to remain creative and adaptable, looking for opportunities to expand knowledge and skills beyond the core team. For example, they successfully applied for funding from the European Union to run three randomised controlled trials (RCTs) to evaluate the effectiveness of new measures intended to help strengthen research and innovation in start-ups and SMEs (you can read more about IGL’s support for this programme here). Securing external funding for this work helped FFG to run more advanced trials than might otherwise have been possible, which in turn helped make the case for expanding this area of work within the organisation.

3. Know when to stop, pivot or progress

A policy experiment has a clearly structured approach to learning - defined before the experiment starts rather than afterwards - and it generates new information, evidence or data. A rigorous policy experiment will have a theory of change, systems and processes in place to capture learning, and a clear timeframe with limits or checkpoints where results will be assessed and decisions made about whether it should be adjusted, scaled up or discontinued. Within the TAFTIE task force, we helped agencies to think about how they could start to develop these systems, and become more rigorous about identifying and testing their assumptions along the way. For example, the Technology Development Foundation of Turkey (TTGV) ran a small pilot within one of their programmes during the course of the Task Force to test a new idea and - all being well - determine whether a larger trial might then be feasible to evaluate its impact. Doing this ultimately saved the agency from investing resources in an intervention that would not have worked despite appearing promising at first - a great illustration of why not going ahead with a programme based on incorrect assumptions should be seen as a big success.

4. Get support from others

After working with dozens of government agencies around the world to help them develop their experimental capabilities, we know how steep the learning curve can be. It is never a linear process, but rather one that tends to progress in fits and starts. On this kind of learning journey, it is invaluable to have a community of peers that can share what they are doing and learn from the achievements and setbacks experienced by others. The TAFTIE task force provided exactly this kind of network, with participating agencies taking turns to give ‘deep dive’ presentations of their work and receive feedback and support from their counterparts in other organisations. It is also a model we use at IGL - where we seek to facilitate relationships between our partner agencies who are working to address similar challenges and support them to develop their capabilities to use evidence and data more effectively. 

To learn more about the partnership and the different ways in which IGL can support your agency to become more experimental, do get in touch! 

If you would like to learn more about the TAFTIE Task Force, you can read the final report here and contact the FFG coordinators directly.