We are providing a preview of a project that analyzes two field experiments with 1,974 software developers at Microsoft and Accenture to evaluate the productivity impact of Generative AI. As part of our study, a random subset of developers was given access to GitHub Copilot, an AI-based coding assistant that intelligently suggests ‘completions’ for code. Our preliminary results provide suggestive evidence that these developers became more productive, completing 12.92% to 21.83% more pull requests per week at Microsoft and 7.51% to 8.69% at Accenture (depending on specification).
IGL Trials Database
IGL curates a database with randomised controlled trials in the field of innovation, entrepreneurship and growth. Browse our list of topics, see it as a map, or use the search function below.
This study investigates the integration of artificial intelligence (AI) in the screening processes of early-stage innovations, traditionally conducted by human evaluators, across various professional and competitive settings. Through a randomized controlled trial involving around 400 participants from the MIT Solve expert internal screener team and from community leveraged startups screeners, this research explores whether AI-assisted human evaluators or AI-only evaluations enhance the efficiency and quality of decision-making compared to traditional human-only evaluations.
Understanding how to design policies to effectively reduce firm-level carbon emissions while minimizing impacts on economic growth is a question of central importance in the battle to mitigate climate change. The EU is proposing a Carbon Border Adjustment Mechanism (CBAM) that will tax imports to better reflect their carbon content. This project evaluates three policies that provide firms with training and assistance obtaining loans with the goal of mitigating the impacts of CBAM on Turkish SMEs.
This study is focused on the relationship between borrowing constraints, access to cutting-edge technology and information about cutting-edge technology on the performance of U.S. online businesses. With the help of two large U.S. technology companies we will be able to randomize access to loans and free cloud computing credits (as well as information about the potential use of technology) to otherwise identical (generally small, but fast growing) firms, to see if they will have a causal impact on firm development.
Measuring the returns of advertising opportunities continues to be a challenge for many businesses. We design and run a field experiment in collaboration with Yelp across 18,294 firms in the restaurant industry to understand which types of businesses gain more from digital advertising. We randomly assign 7,209 restaurants to freely receive Yelp’s standard ads package for three months. The scale of the experiment gives us a unique opportunity to assess the heterogeneity in advertising effectiveness across a variety of business attributes.
As more and more activities in the economy become digitized, analytics and data-driven decision-making (DDD) are becoming increasingly important. The adoption of analytics and DDD has been slower in small-to-medium enterprises (SMEs) compared to large firms, and reliable causal estimates of the impacts of analytics tools for small businesses have been lacking. We derive experiment-based estimates of the effect of an analytics tool on SME outcomes, analyzing the randomized introduction of eBay’s Seller Hub (SH), a data-rich seller dashboard.
Can artificial intelligence (AI) assist human employees in increasing employee creativity? Drawing on research on AI-human collaboration, job design, and employee creativity, we examine AI assistance in the form of a sequential division of labor within organizations: in a task, AI handles the initial portion which is well-codified and repetitive, and employees focus on the subsequent portion involving higher-level problem-solving. First, we provide causal evidence from a field experiment conducted at a telemarketing company.
New AI tools have the potential to change the way workers perform and learn, but little is known about their impacts on the job. In this paper, we study the staggered introduction of a generative AI-based conversational assistant using data from 5,179 customer support agents. Access to the tool increases productivity, as measured by issues resolved per hour, by 14% on average, including a 34% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers.
Can a set of low-cost behavioural nudges encourage more small businesses to adopt productivity-raising digital technologies? This randomised controlled trial sought to test whether businesses could be nudged into using a cloud-based system to improve the efficiency of invoice processing. All participants in the trial were offered access to the system free of charge for a 12-month period, with a treatment group receiving weekly email reminders to make use of the system.
India is host to 63 million Micro, Small and Medium scale Enterprises (MSMEs), contributing to a large share of employment, industrial output as well as high volume of emissions per unit of output. Therefore, adoption of energy efficient (EE) technologies by MSMEs is crucial in improving not only their competitiveness through cost reduction but also worker wellbeing and productivity through improvements in the work environment. Enterprise owners most often do not internalize the benefits of the latter; like productivity gains due to reduction in exposure of workers to heat, pollution etc.
The public release of Large Language Models (LLMs) has sparked tremendous interest in how humans will use Artificial Intelligence (AI) to accomplish a variety of tasks. In our study conducted with Boston Consulting Group, a global management consulting firm, we examine the performance implications of AI on realistic, complex, and knowledge-intensive tasks. The pre-registered experiment involved 758 consultants comprising about 7% of the individual contributor-level consultants at the company.
There is a growing belief that scalable and low-cost AI assistance can improve firm decision-making and economic performance. However, running a business involves a myriad of open-ended problems, making it hard to generalize from recent studies showing that generative AI improves performance on well-definedwriting tasks. In our field experiment with 640 Kenyan entrepreneurs, we assessed the impact of AI-generated advice on small business revenues and profits.
Across developing economies, cash is the conduit for retail transactions. Policymakers, multinational product manufacturers and marketers of electronic payment systems are interested in understanding how to stimulate the growth of electronic payments in emerging markets. In this paper, we investigate what hinders the adoption of e-payment technology by traditional retailers, in particular, whether barriers to adoption are technological, informational or financial in nature.
The limited market size of many small emerging economies is a key constraint to the growth of innovative small and medium enterprises. Exporting offers a potential solution, but firms may struggle to locate and appeal to foreign buyers. A six-country randomized experiment was conducted with 225 firms in the Western Balkans to test the effectiveness of 30 hours of live group-based training and 5 hours of one-on-one remote consulting in overcoming these constraints.
I investigate how firms should design human/AI collaboration to ensure human workers remain engaged in their activities. I developed a formal model that explores the tension between AI quality and human effort. As AI quality increases, humans have fewer incentives to exert effort and remain attentive, allowing the AI to substitute, rather than augment their performance. Thus, high-performingalgorithms may do worse than lower-performing ones in maximizing combined output.
Using a new survey of firms in New Zealand, we document how exogenous variation in the macroeconomic uncertainty perceived by firms affects their economic decisions. We use randomized information treatments that provide different types of information about the first and/or second moments of future economic growth to generate exogenous changes in the perceived macroeconomic uncertainty of some firms. The effects on their decisions relative to their initial plans as well as relative to an untreated control group are measured in a follow-up survey six months later.
The ‘Evolve Digital’ trial was developed with the objective of boosting digital adoption in small family firms through identifying a cost-effective, yet productivity-enhancing programme of peer group learning for small family businesses, which can be replicated throughout the country.
We examine the influence of physical proximity on between-startup knowledge spillovers at one of the largest technology co-working hubs in the United States. Relying on the random assignment of office space to the hub's 251 startups, we find that proximity positively influences knowledge spillovers as proxied by the likelihood of adopting an upstream web technology already used by a peer startup.
This project aims to understand the determinants of adoption of a new technology by firms in Bangladesh's leather goods and footwear industry.
This paper studies the impact of business modernization on the sales performance of traditional retailers. We define modernization as adopting tangible structures and business practices of organized retail chains (for example, exterior signage with store name and logo, or a database to record product-level information). To address our research question, we implement a randomized field experiment in Mexico City with 1148 traditional retail firms.
Expansion of e-commerce presents new opportunities for small and medium enterprises (SMEs) to enter broader market at lower costs, but the SMEs face barriers to growth after entry. To facilitate new entrants to overcome these barriers, this paper explores implementing a training program as a randomized controlled experiment with over two million new sellers on a large e-commerce platform.
The use of machine learning (ML) for productivity in the knowledge economy requires considerations of important biases that may arise from ML predictions. We define a new source of bias related to incompleteness in real time inputs, which may result from strategic behavior by agents. We theorize that domain expertise of users can complement ML by mitigating this bias. Our observational and experimental analyses in the patent examination context support this conjecture.
This article studies the effects of the adoption of artificial intelligence on teams and their performance and coordination in a laboratory experiment. We posit that automation decreases organizational performance, interferes with team member coordination, and leads to behavioral changes in human co-workers. We randomize the introduction of automated players and new hires into "laboratory firms" (Weber and Camerer, 2003) who must coordinate in teams playing a game on the Nintendo Switch console.
This trial proposes to evaluate a model for scaling up affordable access to effective STEM education through national online education platforms.
A randomized control trial in central Tanzania, centered on the production and distribution of a ”Yellow Pages” phone directory with contact information for local enterprises.
The study is an impact evaluation of a training program that induced SMEs to adopt broadband connections, establish presence on online retail and potentially export their goods or services.
In this study we assess the effects of a decentralized extension program and an additional video intervention on the adoption of integrated soil fertility management (ISFM) among 2,382 farmers in Ethiopia using a randomized controlled trial. ISFM should enhance soil fertility and productivity by combining organic and inorganic soil amendments. We find that both extension-only and extension combined with video increase ISFM adoption and knowledge.