New experiments are shaping the way we think about innovation, entrepreneurship and growth

What is the best way to support SMEs adopt Artificial Intelligence technologies? Which nudges encourage employees to develop new ideas within their organisations? Can algorithms improve labour matching platforms? In the 21st century, policymakers face tough challenges, but they also count with a new set of tools from digital to behavioural to tackle them. However, it’s important to test these new projects and gather enough evidence to support policymakers to make more informed decisions.

Some of these questions were addressed at the IGL Winter Research Meeting that took place on Thursday, 13 December 2018 at Nesta, in London. A day packed with six presentations and discussions of design-stage, ongoing and completed randomised controlled trials (RCTs) which spanned the innovation, entrepreneurship and business growth field.

The six experiments presented covered a wide range of sectors and geographic areas, as well as the use of different experimental tools, from nudges to new management practices in decision making. The research meeting is a forum to discuss design-stage, ongoing and completed academic research that uses experimental methods to improve our understanding of the drivers of innovation, entrepreneurship and growth, and potential interventions to accelerate these. The IGL Winter Research Meeting was supported by inspiring discussants from highly experienced and prestigious organisations in the field of policy experimentation, including the World Bank and the Behavioural Insights Team.

The main questions presented and discussed during the IGL Winter Research Meeting were:

1. How does managerial framing affect intrapreneurship? Using behavioural insights, Coen Rigtering from Utrecht University presented an IGL-funded trial inviting employees to submit proposals within their organisations. They tested how different registration strategies and provision of examples would affect the number and quality of submitted ideas. Results show that opt-out registration increases participation without reducing quality. On the other hand, whereas providing examples enhances the usefulness of ideas presented, it may decrease the novelty and number of total submissions.

2. Does business coaching increase new technology-based firms’ survival? There is scarce evidence concerning the causal effects of experienced coaching on venture survival. Nicolai Heinzelmann from the IST Institute presented another IGL-funded trial intended to get a deeper understanding of the impact of coaching activities and how they may enable accelerators and entrepreneurship programme providers to optimise their activities and foster firms’ survival and business growth.

3. Can we encourage SMEs to adopt new technologies? Anna Valero from the London School of Economics presented a design-stage RCT measuring the impact of different strategies to encourage retail and hospitality SMEs to adopt Artificial Intelligence technologies, in particular, chatbots and marketing automation. The trial will address different barriers such as lack of information, motivation or financial constraints by comparing a light touch intervention with a more targeted approach.

4. Does promoting a scientific approach to decision-making increase productivity? Chiara Spina from Bocconi University and Elena Novelli from Cass Business School presented three linked trials, one at the design stage, that measure the impact of using a systematic approach to decision making on SMEs’ productivity. Business owners using a scientific approach may increase precision in information gathering and analysis, and therefore select strategies that increase their productivity. Their design stage trial builds on the rigorous evidence produced by a pilot RCT conducted with new firms in Italy and funded by IGL.

5. What is the effect of alleviating information frictions using an algorithmic matching platform? Working with a large job placement platform in South Africa, Lukas Hensel from the University of Oxford presented an intervention that may improve matching quality between work seekers and vacancies using algorithms based on observable characteristics. The design-stage trial seeks to assess if algorithmic matching leads to better business outcomes for firms, reduces hiring costs and increases the inclusiveness of labour markets.

6. How to address biased beliefs in scientific careers outcomes? US chemistry PhD students are overly optimistic about the state of the academic market and their prospect career outcomes. Patrick Gaule from the University of Bath wanted to measure if providing accurate information on historical placement records changes individual career aspirations and outcomes. Early results show that the information intervention led to an adjustment in beliefs, but not to changes in career intentions.

All these trials and the other projects discussed informally during the meeting provided an excellent overview of new pieces of evidence that are produced in nascent areas such as organisational behaviour. Ongoing trials benefited enormously from the feedback received from the discussants, such as suggestions for innovative ways to assess technology adoption by using web scraping. As Anna Valero from the London School of Economics highlighted, “presenting our proposed project at the IGL Winter Research Meeting gave me the opportunity to gain valuable feedback from experts in the field, which we will now build upon as we take our work forward.’”

The meeting also provided clear examples of the challenges of designing successful trials. For instance, Nicolai Heinzelmann from the IST Institute exposed how their trial suffered some difficulties that affect their assessment of the results. This provided a very useful opportunity to discuss how researchers could work better with delivery partners or reduce non-compliance or low response rates.

“It was also incredibly useful to hear about other projects and meet a network of researchers working on RCTs in different contexts. I highly recommend participating in future IGL events,” said Anna Valero. If you want to know more about the lessons learned, don’t miss IGL Principal Researcher Triin Edovald’s blogpost outlining the five key takeaways from the Winter Research Meeting.

Supporting researchers to engage with policymakers

On Wednesday, 12 December 2018, ahead of the IGL Winter Research Meeting, IGL organised a workshop for PhD students. Researchers received an overview by the IGL team on how field experiments are being used in the fields of innovation, entrepreneurship and growth.

IGL’s Director Albert Bravo-Biosca presented a series of examples that framed the rest of the day and highlighted how RCTs can be an efficient tool, not only to assess the impact of a particular programme but also to explain individual behaviour. For instance, some trials have successfully improved the collaboration among researchers by using simple nudges, such as the Harvard Medical School’s work with biomedical researchers. They were also provided with some of the pressing challenges policymakers face as well as basic insights on designing field experiments which they later used to develop trial ideas based on their own research.
PhD students had the exclusive opportunity to present their trial proposals to representatives from the UK government, the Norwegian government and the World Bank. The dialogue with policymakers provided very valuable insights on how to improve the quality and impact of their research.

To end the day, David McKenzie, Lead Economist in the Development Research Group at the World Bank and author of the world-renowned Development Impact Blog, provided a keynote on practical issues in doing field experiments. The session covered how to persuade policymakers to experiment, how to implement programme logistics properly, the best ways to adequately measure firm outcomes, and choosing the right time frame to measure impacts. Some of his ideas were also mentioned during the Research Meeting, like the implications of the rule of inverse-square for programmes with low take-up.   

The IGL Winter Research Meeting was an excellent opportunity for researchers and policymakers to be more experimental, learn from each other and strengthen ties in the innovation and growth sector. IGL will organise the IGL2019 Global Conference on 21-23 May in Berlin, Germany: don’t miss this opportunity to join the conference and discuss new trials that are shaping the way we think about innovation, entrepreneurship and growth.