Since launching the Unlocking Innovative Potential: Experimentation Programme at the start of the summer, we have been busy kicking off the first phase of work to bring together our community of practice to uncover experiment ideas. The Unlocking Innovative Potential: Experimentation Programme is more than just a collection of pilots looking to understand how the UK might maximise efforts to diversify and increase skills and talent within R&D sectors. This programme has been designed as a pilot itself. We are testing a new, collaborative approach to identifying evidence gaps and designing targeted experiments, with particular focus on testing if randomised controlled trials (RCTs) would be useful in this space. Activating our delivery partners and expert researchers by introducing them to their role was of paramount importance to get going quickly. Through this first phase, we’ve learnt of the importance of opening space for both peer learning on experimental methods and individual deep dives on context-specific barriers and enablers to experimentation with each organisation.
Setting the intention
From the outset, our goal was to build a community of practice geared towards generating and using evidence to inform decisions on what works to increase and expand R&D skills and talent. All of our seven delivery partners became part of our Programme’s cohort because they work on a number of existing programmes related to the policy challenge. Our expert researchers had also engaged in research activities focused on different aspects of the innovation and skills gap agenda. By setting up this community of delivery organisations and researchers, Innovation Growth Lab (IGL) hopes to make connections that ensure opportunities for further experimentation aren’t tied to this programme alone. For example, while our ambition within the year would be to test whether experimental methods can indeed be applied to existing policy-relevant programmes, developing pilot experiments into full RCTs would take much more time.
By matching researchers to policy-relevant programmes with the potential to generate useful evidence, pilots that prove to be successful in this year’s programme might continue to be developed within existing UK research institutions. With this in mind, IGL might then successfully become a vehicle driving experiment ideas, but with the wider community developing potential ideas further. Wider scaling is woven across the research and innovation ecosystem, rather than dependent on our small organisation alone.
Getting ready for action
Our first introduction meeting brought together all delivery partners to present the aims of the Experimentation Programme, how the funnelled approach to developing ideas would be tested, and the purpose of the Community of Practice. This meeting was the first time delivery partners were able to meet one another and get a better sense of what they might expect from the Community. While this introduction was well received, with so many partners it was difficult to dive deeper into the specific contexts of each, and IGL needed a deeper level of knowledge on each partner to onboard the expert researchers in a useful way.
The second encounter, set up as the start of the Scoping Phase, brought delivery partners together again - but this time allowed for space within smaller groups to discuss the details of each prospective programme that may embed experimentation within it. In pairs and with the facilitation of an IGL team member, partners mapped their target population, the intervention idea for the experiment, what they might use as a point of comparison to understand the impact of the intervention, and the expected outcomes. In this session, the aims were to:
- Support delivery partners to try-out rapid experiment ideation, to better understand the constraints of experimentation versus other ways of learning what works that they might have been more accustomed to
- Continue to facilitate peer learning through the exchange of listening to another organisation tackle the same activity within their context specific boundaries
- Give IGL staff a better sense of the potential for experimentation within each organisation.
Prior to the scoping sessions, individual partners were sent a list of questions to help them prepare, alongside a short information pack about the experimentation methodology defined by IGL.
Observing and learning as we go
The mix of both individual, pair and group engagements worked well to accommodate different partner needs, but as IGL came to the point of needing to report on scoped ideas to researchers, who had an interest in ideas with the potential to scale into large field experiments, there were still many open questions. For example, we were able to quickly get a sense of interesting experiment ideas (10 in total), such as:
- A messaging trial to increase and improve engagement with educators using an online AI platform to link students to science/innovation founders;
- Testing the impact of one-off engagements vs multiple encounters with interventions imparting STEM skills over time;
- Testing the effect of soft-skill and confidence training as a complementary (additional) option to employability-skills training.
Ideas of this kind were useful but we still needed to know more about details such as who the target groups were, if partners had access to a large enough sample size or if they would be open to randomisation. We had expected some delivery partner organisations to need more time to feel where experimentation would be best suited within their programmes of work, but we had also hoped to have a few more front runners easily identifiable. Instead, at the end of the scoping phase, we had many potential experiments that might align with wider research and policy questions.
There was also little time for partners to get to know each other on a deeper level. Our programme brings together partners working on a broad range of ways to unlock innovative potential, from interventions in schools to expose more young people to innovation skills through to accelerator programmes to support founders of diverse backgrounds to scale. We needed more time to identify spots of common interest, and for partners to connect. We were able to do some of this, for example identifying an experiment idea on design skills which closely matched the research expertise of one of our researchers - but we sense there were many more connections that could be made!
With these lessons, we decided to extend the scoping phase of work throughout the project life-cycle. Through the Community of Practice meetings, IGL can allow partners to have more space to learn and dialogue with one another. Our aim continues to be to facilitate connections and as soon as experiment ideas are sufficiently developed, they can move into the ideation phase of our programme where expert researchers alongside IGL set the basis for pilot RCTs to be designed.
Taking a portfolio approach, more one-on-one engagements with delivery partners and researchers also help us tread the line between aiming to answer the most pressing research and policy questions, and ensuring delivery partners can also benefit from the evidence generated (which comes with significant investment on their part).
Finally, as our scoping phase does not come to a close but instead moves into a second iteration, we will begin to match researchers with aligned questions to partners with programmes that might benefit from access to the target populations or interventions being implemented over the next year. If you are a researcher who is interested in sharing your research focus area, please consider joining IGL’s research network or alternatively look out for our next scheduled updates.