The promise of user-centred design
One of the key competitive factors for businesses is the capability to develop more valuable products and services. In traditional businesses, this is normally done through R&D, engineering, marketing, as well as industrial design, which are well established practices.
With the digitisation of products and services, methods and practices for developing products and services in companies have drastically changed. Research and development processes have become less structured, and time to market has decreased substantially. More recently, personalisation and mass customisation of products have become a must.
New methodologies and practices have emerged to cope with such industry and market transformations: prototyping has become less expensive and new, and less structured product development methods have emerged (e.g. Agile). Understanding the needs of customers and end users, as well as their capability to use ever more complex products and systems (e.g. software programs), has become increasingly critical.
The usability, acceptability and appeal of a digital product – which together can be referred to as 'user experience' – is currently a key factor in determining customer choice, and therefore product market profitability. However, good user experience stems from good design. And, given the high level of interactiveness of digital products and services, good design is not only required for the product materiality itself, but also for the ways in which the users interact with the product and manage to use its functionalities to pursue given goals. For these reasons, good design for digital products requires having good ‘interaction designers’.
In order to maximise usability, interaction designers utilise user-centred design methods and techniques during the early stages of the development of a new product. One of these is the ‘Design Sprint’, a practical method developed by Google Ventures that applies design thinking principles (such as customer interviews and user testing) to support digital startups in finding solutions to product design problems.
Methods such as the Design Sprint have been shown to substantially improve the quality and outcome of digital product design. However, SMEs are often not aware of the added value of these techniques and are not equipped to adopt them.
The User Experience Challenge (UX Challenge) is an open innovation initiative that allows SMEs to learn the benefits and feasibility of using user-centred design methods such as the Design Sprint during the development of a new product. The UX Challenge is structured as a competition between teams of young designers (typically, university students specialising in human-computer interaction) that team up and collaborate for one week to find a solution to a design problem regarding a digital product presented by one SME. The teams carry out a full Design Sprint (including user testing), with the purpose of delivering a validated prototype solution to a user experience design problem. Examples of solutions include redesigned user interface mockups, user flow wireframes, and interactive interface prototypes. Human-computer interaction researchers and UX design professionals also take part as team mentors.
Until recently, it was not known whether UX Challenges are effective at increasing SMEs’ readiness to adopt user-centred design methods. This is exactly the point where the 200SMEchallenge project kicked in.
The 200SMEchallenge experimentation
The 200SMEchallenge project involved carrying out a UX Challenge and using a randomised controlled trial to evaluate its effects on SMEs’ readiness to implement user-centred design methods. This project was one of 13 funded by the European Commission under the Horizon 2020 programme, INNOSUP-06-2018.
The project builds on the longstanding experience of Hub Innovazione Trentino, which has run five editions of the UX Challenge in Italy since 2017. The 200SMEchallenge was replicated in six other European countries (Germany, Finland, Lithuania, Spain, Estonia and Denmark). Nearly 200 SMEs were recruited, of which 60 received access to the UX Challenge and the others served as a control group. Because the access to the UX Challenge was strictly dependent on randomisation, the two groups of companies were, on average, equivalent and hence comparable.
Main results
Three weeks after the UX Challenge events, participating SMEs were asked to fill out an online survey, including a selection of questions about their digital design readiness and awareness.
In line with the expectations, treatment group participants showed a 19% higher knowledge of Design Sprints than the control group, and a 12% higher knowledge of how to implement digital design. The treatment group also had a 6% higher self-perceived general knowledge about methodologies such as user centred design, design thinking and Design Sprint – although this difference was not statistically significant.
There are also indications that the UX Challenge also had positive impacts on participants’ attitudes toward digital design (i.e., the extent to which they believe that their companies would benefit from applying the Design Sprint). However, this difference is again not statistically significant.
No effects were detected on UX Challenge participants’ aspirations and expectations that their companies will invest in digital design and/or adopt digital design techniques in the next 6 to 12 months.
The effects of the intervention
Lessons learned and next steps
To the best of our knowledge, this is the first and only experimental study on the impact of an innovation contest on user-centred design offered as a service to digital companies with the aim of increasing digital design readiness and awareness.
The results of this randomised controlled trial suggest that the UX Challenge is a promising way to improve participants’ objective and practical knowledge about a Design Sprint and digital design.
The evaluation also suggests that the discrepancy between the largely positive impacts on knowledge (and the tentative positive impacts on attitudes) and the zero impacts on aspirations and expectations to adopt a Design Sprint could be explained by some organisational and financial constraints faced by the companies.
Future studies are needed to consolidate these findings in at least three ways. First, more research is needed to understand whether (and under which circumstances) the increased knowledge about digital design leads to concrete, tangible changes in companies’ approaches to digital design. Second, this study suffered from a small sample size and from a differential response rate in the online survey between the treatment and the control group. A number of statistical checks and a range of different impact estimation approaches have been performed, and these provide further support to the above presented findings. However, future studies in this field should assign highest priority to experiment designs or incentives mechanisms aimed at increasing the number of participating companies. Third, replication studies would be useful to test the intervention in other contexts beyond those analysed in this study and in times that are not heavily affected by the disruption induced by the pandemic.
Info on the project
To find out more about the project: https://www.200smechallenge.eu/
The study was preregistered at the AEA RCT Registry: https://doi.org/10.1257/rct.6246-1.0
In 2018, the European Commission introduced a new EU Horizon 2020 programme - INNOSUP-06-2018 - to encourage innovation agencies across Europe to experiment with their policy programmes. Here at the Innovation Growth Lab, we’ve been supporting both the EU and innovation agencies. This blog presents the findings from one of the selected projects - for further details click here.