IGL is developing a new experimentation toolkit to help policymakers in innovation and entrepreneurship test new ideas. The article below, which will be part of the toolkit, is a primer on what trials are.
Trials are powerful, yet the concept that makes them work is simple…
In essence, a randomised controlled trial (RCT) is a trial or experiment carried out on two or more groups to capture the impact of an intervention where participants are randomly assigned to receive an intervention or not.
Randomised controlled trials, or simply trials, can take many forms. The most straightforward examples is that of a two-group approach. Participants are randomly assigned to either an intervention group (also called a ‘treatment’ group) who are given the intervention, or a control group who are not1.
Each group is tested at the end of the trial and the results from the groups are compared to see if the intervention has made a difference and achieved its desired outcome. If the randomised groups are large enough, you can be confident that differences observed are due to the intervention and not some other factor.
This is because the randomisation ensures that participants in both groups are similar in characteristics - including ‘unobservable’ ones. Any residual difference should therefore be random.
The following image illustrates the key principles of trials for this simple randomised design:
Trials provide gold standard evidence…
Trials are considered the ‘gold standard’ for establishing a causal link between an intervention and change. More specifically, trials are considered to be the most rigorous way of establishing if evidence resulting from an experiment shows that the outcomes have been caused by the programme or intervention – known as ‘causal description’ (or ‘descriptive causation’).
This is why there is a strong rationale for greater experimentation and use of trials in IEG policy.
…and have the flexibility to answer a wide range of policy and research questions…
Trials are very flexible. For example, one could add more than one treatment groups to test the impacts of different types of treatment. The ‘pure’ control group could also be lost, with the trial only used to compare differences between two forms of treatment.
…but whilst the concept is simple, careful planning is required to ensure trials are used effectively
There are many elements that need to be considered when designing and delivering trials in a policy setting.
For example, it is mentioned above that the size of randomised groups need to be large enough to ensure participants have a similar mix of participants. What is ‘large enough’ can vary immensely between trials depending on the nature of intervention being delivered, the characteristics of participants and practical issues such as data collection.
Our toolkit is designed to help you understand and deal with all of these factors so that you can effectively design, deliver and interpret the results from trials.
The toolkit contains a ‘When' section describing what you need to consider when identifying opportunities for trials, as well as a ‘How' section providing a step-by-step guide for designing and delivery a trial.
The toolkit will officially be launched at IGL’s Global Conference on June 13-14; it is thought of as a living resource, which will be updated with new content and answers to Frequently Asked Questions. The conference will also be an opportunity to discuss trials and experimentation, as well as new trends and cutting-edge ideas in innovation and entrepreneurship more generally, together with senior policymakers and practitioners from innovation agencies around the world. You can read the programme here and register for the event.