In the last few years, with my colleagues Alfonso Gambardella and Arnaldo Camuffo at Bocconi University (and colleagues from other institutions around the world), we have been running a series of randomised controlled trials (RCTs) to understand the impact on entrepreneurial performance of teaching entrepreneurs to operate like scientists when making decisions.
I led one of these RCTs in the UK, conducted at Bayes Business School (City, University of London), carried out under the UK government’s Business Basics Programme. The intervention was embedded in a business support programme in which entrepreneurs attended training sessions and events for one year. A randomly selected subset of the entrepreneurs were taught how to make decisions using a structured approach to decision making, while a control group received similar training content but without the scientific approach. The participants were regularly monitored throughout the programme, and the scientific training was found to have significant effects on their decisions and on their business performance.
The RCT achieved good results in terms of recruitment and retention of participants, and the underlying research has received multiple awards of recognition in the field. Based on this experience, what tips do I have in terms of organisation and logistics for anyone who is considering running an RCT?
I believe that it is very important to consider the triple nature of such a project. An RCT is, first, a research project; second, it is an instrument to deliver impact to the user community; third, it is a temporary pop-up organisation that needs to be designed (often from scratch) and maintained in the most effective and efficient way. Each of these three components requires attention.
First, let’s look at the research component. In our project, we invested much effort in making sure that the research design was executed to the highest standards. We carefully designed the intervention, based on a detailed protocol, collecting feedback from experts in the field (first and foremost IGL). The delivery of the intervention was very meticulous. Our instructors were recruited carefully, we trained them to teach the programme in a consistent way, we tested them to make sure they were comfortable with the material, we supported them in their delivery, which we monitored to ensure consistency and provided relevant feedback along the way. A similar process was followed in hiring and training the team of research assistants who assisted and interviewed entrepreneurs during the programme. We always made sure everyone in the team was familiar with the relevant aspects of the protocol and introduced regular check points and quality control procedures. This ensured that the relevant research standards were met and results would be reliable.
Just as important for the success of an RCT is the second component, delivering impact to the user community – in our case the entrepreneurs themselves. This requires taking their perspective at all steps of the process. For instance, during the RCT design phase, we did not only devote time to the design of the intervention but also to the creation of a fully-fledged programme that would deliver maximum value for the time the entrepreneurs put in. Whereas the intervention lasted for three months, our RCT required entrepreneurs to provide data for about 10 months. To provide good value for the remaining seven months, we offered entrepreneurs activities that they would find useful, such as networking events, mentorship sessions and dedicated lectures (the latter were delivered in an identical way to the treatment and control groups, to avoid biasing the results of the experiment). We committed to maintaining an open dialogue with entrepreneurs throughout the programme and answering their questions and concerns, such as providing logistical information or information about data privacy and processing. We always felt it was incumbent upon us to be as clear as possible. We strived to provide the best customer service, despite our limited resources, by listening to suggestions and trying to accommodate requests, as long as they would not compromise the research design.
The third aspect of running an RCT is that it implies setting up a temporary organisation within a short time window. This implies, for instance, accommodating hundreds of RCT participants (almost 300 in our case), hiring dozens of people and designing dedicated logistical procedures. Except for a few organisations that run RCTs regularly, and which therefore can afford to have dedicated permanent personnel throughout the year to work on RCTs, any other organisation does not have – quite understandably – idle resources to accommodate the extra work required. To address this issue, it was very important to liaise as soon as possible with our colleagues in the procurement, HR, security, facilities, audio/visual, and IT teams and to explain clearly what we were trying to achieve and why the project was important for the school as a whole. We made sure we understood what their constraints in terms of procedures were, and we did our best to work around those constraints, minimising requests for changes. Where a change was necessary, we made sure we clearly communicated the problem and offered to work together to minimise the disruption.
Another challenge was building a team of research assistants who had the right competences and the right incentives to work on the programme. One of the challenges was that our project lasted for less than a year, so we would not be able to recruit those who were already in employment. Instead we recruited university students. Identifying individuals with the right competences, motivation and career aspiration was a key success factor and we deliberately designed roles customised for them so that they could thrive. For example, some members of the team had a talent in communicating with entrepreneurs, others loved being involved in the backend and using their coding skills to monitor the data collection and cleaning, others were very gifted in monitoring and motivating the rest of the team. We designed a hierarchical team structure with clear roles and responsibilities for everyone. We elaborated, communicated, and monitored clear procedures for each job. This was essential to ensure the highest level of consistency and respect for procedures that an RCT design requires and the highest research standard. Each member of the team was aware that autonomous deviations from the procedure were not allowed. However, any member of the team was very welcome to approach me with suggestions if they realised that a procedure required or would benefit from improvement. We would talk about the pros and cons and then test the new procedure and scale it. This happened very often and led to substantial increases in effectiveness and efficiency as the various members of the team had valuable insights in the different areas of the project. In fact, I found it extremely rewarding every time a member of the team took the initiative of identifying, proposing, and implementing a relevant improvement and saw it as a sign that we were working in the right direction. We also devoted time to mentoring the team so that they would make the most of this experience and use it to contribute to their career aspirations.
Of course, we also hit a few bumps along the road. When this happened, we recognised the problem, we investigated what happened very closely, we learned from it, and we used this insight to improve the process.
I will be presenting the results from this experiment at IGL’s online event on 13 July 2022. Click here to register and hear more. To find out more about my work, click here.
This project was funded by the Department for Business, Energy and Industrial Strategy and implemented with the support of Innovate UK and IGL under the Business Basics Programme. IGL also supported an earlier trial of this approach with startups in Italy.