In 2018, the European Commission introduced a new EU Horizon 2020 programme - INNOSUP-06-2018 - to encourage innovation agencies across Europe to experiment with their policy programmes. Here at the Innovation Growth Lab, we’ve been supporting both the EU and innovation agencies to succeed. This piece explores the journey of Anna Koktsidou, a member of Greece’s Business and Cultural Development Centre (KEPA) team, who are currently partaking in the INNOSUP programme.
Anna has worked for KEPA for over a year, and for her, their EU funded project ‘Design Customised Support for Innovative SMEs’ has the potential to demonstrate the famous adage: teach a man to fish and you feed him for a lifetime. She sees imparting the concept of design thinking - a human-centred approach to innovation - as a way of granting businesses the opportunity to intrinsically shift their approach and consistently yield greater long-term positive outcomes.
As the INNOSUP-06-2018 programme began, KEPA was launching the Hellenic Design Centre in Thessaloniki as part of wider efforts to promote the use of design thinking in both the public and private sector.
Design thinking allows organisations to approach problems in a human-centric way, researching and defining their users’ needs, challenging assumptions and creating innovative solutions as a result. KEPA hopes to grow the use of this methodology within Greece, in order to enhance the competitiveness of Greek SMEs with the production of innovative products and the creation of high-quality services.
An earlier H2020 project, Design Shots, allowed the team to explore a light-touch design diagnostic for SMEs. Later, the INNOSUP-06-2018 programme afforded KEPA with an exciting opportunity to create a more substantial policy intervention that would target an apparent gap in the support available to SMEs after they had drawn down innovation funding. (You can read more about the intervention they have developed here).
Running the pilot as an experiment to demonstrate the effectiveness of design thinking was an idea with strong buy-in from leading government figures in Greece, as the majority of their information on the impact of design thinking was derived from other countries and large corporations. For Greek officials, there was a demand for more evidence which was based on the specifics of their environment, factoring in the smaller nature of some Greek organisations and their idiosyncrasies.
Though KEPA had previously been unfamiliar with experimentation in general, and typically planned evaluations once programmes were underway, they now aimed to test the impacts of design thinking by using a small-scale pilot with some participants randomly selected to receive the full ‘Design Customised Support’ programme. In doing so, the team’s objective was to build their own capacity to run experiments and help embed this approach across innovation policy.
The unforeseen impacts
Anna’s own experimental journey with KEPA began as the project was just beginning, and it saw her covering implementation, recruiting and attracting SMEs to take part, but also developing KEPA’s evaluation plan.
This was not the first time Anna had come across this human-centred approach; she had worked in a communications position for the Joint Secretariat team of Interreg prior (an organisation helping regional and local governments across Europe to develop and deliver better policies). Anna has since realised that a large part of her previous work centring on an audience’s behaviors falls into what she now considers to be design thinking.
A number of challenges were encountered as KEPA began refining the details while developing their planned experiment. As an entirely new intervention and with support to be tailored to business needs, it proved difficult to know in advance how best to define outcomes measures and collect data in a way that would enable statistical comparisons across the two arms of the trial. With ten SMEs able to participate in the actual pilot, however, the achievable sample would always have been too small to provide reliable estimates of the programme's impacts. The team therefore had to adapt plans, shifting the focus to more qualitative assessments of how SMEs would benefit from the programme and creating the basis for future quantitative evaluation.
Initially when the call opened, optimism was high within the team; there was a significant level of SME interest, with numerous organisations seeking more information. However, soon after launching this phase of recruitment, KEPA’s experience was put to the test by the unprecedented spread of the Covid-19 pandemic within Europe. As a result, it was unclear not just as to when was best to resume activities, but whether proceeding at all would be viable.
The call had to be relaunched, and this time around, interest was - as expected - not as high. KEPA’s offer was no longer as appealing to companies dealing with the devastating impacts of Covid-19, who were primarily concerned with staying afloat.
They had hoped to try and capture information from a much larger sample of interested SMEs and then select ten core participants from within this group to receive access to their ‘Design Customised Support’ programme. The small sample size and unusual timing of the call limited the team’s ability to be selective with who entered the pilot and to draw insights on wider demand. There was also a need to overhaul the approach to delivering the intervention - moving to an entirely digital route.
Outcomes and next steps
At the very outset, Anna was hesitant about the benefits of experimentation. She had never seen an experimental, scientific approach applied to social sciences, and the thought of it was somewhat intriguing yet novel. Through running the experiment, however, Anna has admittedly shifted her initial stance rather drastically, having learned some valuable lessons in both what works and what does not when it comes to policy experimentation.
More broadly, Anna attributes part of her learning journey to her participation in the activities organised as part of INNOSUP-06-2018 by IGL. ‘I have to say that the peer learning sessions were very helpful’, Anna notes, ‘especially being guided through the difficulties of implementing the project’. Learning from the experience of others undertaking experiments similar to theirs was incredibly beneficial for KEPA in knowing how best to approach their project.
Though KEPA did not get to explore their hypothesis to its fullest extent given the wider context, Anna can now better recognise the potential of experimentation. The full benefits of running an RCT would come at the next stage of the intervention’s development - at a larger scale and with proven measures. However, with hindsight Anna believes that their pilot could have yielded more valuable evidence if they were able to be more discerning about who took part, selecting SMEs of a similar size, sector and particularly ones facing similar problems.
KEPA has navigated and overcome the challenges faced, and the fact that this was a small-scale pilot has provided them space to develop their approach and learn not only about taking an experimental approach, but also how businesses would use and benefit from the Design support. The pilot granted Anna and her colleagues direct engagement with the businesses and the ability to observe benefits of the more tailored support, gaining greater insight and understanding of the underlying mechanisms and how one could create more quantitative outcome measures to assess outcomes from a larger project.
"The fact that this was a small-scale pilot has provided them space to develop their approach and learn not only about taking an experimental approach, but also how businesses would use and benefit from the Design support."
With the feedback and support of IGL in designing a survey, Anna is keen to begin this next stage of the project, organising interviews with SMEs involved.
Anna hopes the SMEs’ feedback will be beneficial enough to advocate for the design thinking methods used, as she herself is a convert to experimentation. She is keen for experimental methodology to be considered in future funding opportunities and programmes, and has even specifically advocated for its use on a separate project that she is currently working on. The key learning KEPA hopes to take forward for future pilots is to avoid vague parameters and unclear indicators, and instead to be more specific with what they test, using more scientific methods to draw significant conclusions and robust information.
Running a pilot before proceeding to the full trial has been game-changing for KEPA, as this preparation allows for more time-efficient, well-prepared activities later down the line; they have a better sense of best practice and how to avoid potential pitfalls. For the team, the unforeseen barriers to experimental success have only served as further proof of what is potentially possible: this is just the beginning of their experimentation journey.