Creative Credits: A Randomized Controlled Industrial Policy Experiment

This report examines the results of a pilot study, which used a method of evaluation called randomised control trials (RCTs) to see if a popular business support scheme called Creative Credits worked effectively. The pilot study, which began in Manchester in 2009, was structured so that vouchers, or 'Creative Credits', would be randomly allocated to small and medium-sized businesses applying to invest in creative projects such as developing websites, video production and creative marketing campaigns, to see if they had a real effect on innovation. The research found that the firms who were awarded Creative Credits enjoyed a short-term boost in their innovation and sales growth in the six months following completion of their creative projects. However, the positive effects were not sustained, and after 12 months there was no longer a statistically significant difference between the groups that received the credits and those that didn’t. The report argues that these results would have remained hidden using the normal evaluation methods used by government, and calls for RCTs to be used more widely when evaluating policies to support business growth.

Policy implications

Creative credits can dramatically increase the likelihood that SMEs carry out innovation projects with creative agencies, and increase the likelihood that these firms will introduce process or product innovations, and have greater sales. These effects are not found, however, 12 months after the programme's completion.

Additional information

Hypotheses/research question

Can Creative Credits help firms overcome inertia and risk aversion leading to higher levels of collaboration between firms and thus innovation?

Study design

Parallel-group

The scheme was piloted in the Manchester City Region. It was open to SMEs in city of Manchester or one of 13 surrounding boroughs. SMEs from almost every sector were eligible except for those in primary and creative industries. The SMEs had to have fewer than 250 employees, less than £46m in turnover at time of application, be either limited liability, limited liability partnerships, general partnerships (added wave 2) or industrial or provident societies, and registered for VAT. 150 credits were available with intention to distribute equally between two waves (opened for applications in September 2009 and February 2010 respectively). The scheme was promoted through a number of channels such as a media PR campaign, business networks, and outbound telemarketing though it was encouraged to randomise the marketing process to avoid self-selection bias. Online applications were checked for suitability by Nesta project manager before firms were randomly assigned to the treatment group or control group. Companies house data suggested that 4,200 firms were eligible to apply by application - only 1 in 6 did.

Cost benefit ratio

The average project cost £5,400 in total. The firms reported an average of £2,900 in additional sales in year one, and in the next 12 months an additional £7,740. This was expected to continue for 2.5 years. The authors calculate a suggested benefit-cost ratio of 1.37.

Rounds of data collection

4 quantitative Surveys: Baseline + Follow-ups at 6 months (with treatment and control groups, and creative service suppliers), 12 months and 24 months (with treatment and control groups). 4 rounds of qualitative data collection: stage 1: semi-structured interviews with 16 creative service suppliers and 25 SMEs, stage 2: telephone interviews with 22 SMEs and 14 creative service suppliers, stage 3: 7 SMEs participated in two 'Journey Making' workshops, stage 4: semi-structured interviews with 19 SMEs and 8 creative service suppliers.

Baseline data collection and method

Quantitative survey was administered immediately after the lottery took place.

Data collection method and data collected

All participants received quantitative surveys, some participants took part in semi-structured interviews, telephone interview and two workshops.

Unit of analysis

Business

Stratification

Reference

Bakhshi, H., 2013. 'Creative Credits: A Randomized Controlled Industrial Policy Experiment'. Nesta.

Creative Credits: A Randomized Controlled Industrial Policy Experiment

Key information

Additional information

Key facts

Update trial