Please use this form to submit your study for inclusion into our database. It will be checked by a member of the Innovation Growth Lab team, who may be in contact to ask for more information. Your email address * Your name * Title * The name of the study Short summary Workers who sort into institutional settings they prefer may work twice (or many more times) as hard in these preferred settings. This productivity effect is especially important in institutional settings where a taste for competition is strongest. A brief description of the project's goals and its current state Abstract <p>Incentive regimes not only shape workers’ incentives, but also induce worker sorting. Past research finds that workers sorting into high-powered, contingent or competitive regimes tend to be relatively productive. The reasons for higher productivity, however, have not yet been fully understood. These sorted workers tend to have higher raw skills, but also differ on attitudes, preferences and behavioral inclinations. This paper presents results of a field experiment which allow us to precisely distinguish the effect of sorting on the basis of preferences and attitudes — “tastes” for the regime — from the effects of sorting on the basis of raw skills. Further, we contrast these sorting effects with that of varying formal cash incentives. The experimental context is a competitive on-line contest in which elite software developers solve a challenging algorithmic problem over 10 days, competing in independent groups of 20. The effect of sorting on taste, holding raw skills constant, was large. Problem-solving performance doubled when comparing sorted workers with those who were precisely matched on skills but otherwise unsorted. The effect is entirely explained by the higher effort (hours worked) of those sorting on taste, rather than unobserved skills. This effect was roughly the same magnitude as varying the cash prize of a group from $0 to $1000. The effect of skills-based sorting was an order of magnitude smaller than that of sorting on taste. Thus the behavioral implications of sorting were considerably larger than the “compositional” implications of sorting on raw skills and human capital in the classical sense.</p> The full abstract of the study, if available Links https://www.researchgate.net/publication/228138820_High_Incentives_Sorting_on_Skills_-_Or_Just_a_Taste_for_Competition_Field_Experimental_Evidence_from_an_Algorithm_Design_Contest Links to any published papers and related discussions Authors * Affiliations Academic and other institutes that the authors of the study are members of Delivery partner Organisations involved in delivering the trial, if appropriate Year Year Year199419951996199719981999200020012002200320042005200620072008200920102011201220132014201520162017201820192020202120222023202420252026 Month MonthJanFebMarAprMayJunJulAugSepOctNovDec Day Day12345678910111213141516171819202122232425262728293031 Journal Journal publishing the study, if available Publication stage * Working Paper Published Ongoing Research Forthcoming Discussion Paper Research theme * Entrepreneurship Innovation Business Growth Country Country or countries where this study took place. Topics What sort of topics does the study cover? Sample attributes Hypotheses / research question Does a "taste for competition" impact effort and performance in a tournament setting, beyond the presence of skills and formal incentives? Do behavioural factors like overconfidence, risk aversion, high self-regard, feedback aversion, and more generally preferences for competition play a role in decisions of individuals to participate in a work context and ultimately affect performance and effort. Sample Trial population and sample selection 516 individual participants on the TopCoder contest on-line platform. 264 of these individuals who were asked stated that they preferred a competitive regime over a cooperative option, and were randomly assigned to fill up 13 virtual "rooms" of 20 individuals (sorted group). The remainder of the individuals were randomly assigned to different rooms, irrespective of preference for competition. Rooms in both groups were randomly assigned to compete for cash prizes. Number of treatment groups Size of treatment groups 264 individuals Size of control group Unit of analysis Clustered? Yes No Cluster details Trial attributes Treatment description 10-day field experiment in which over 500 elite software developers from around the world generated solutions to a complex algorithmic engineering problem while working in a competitive contest regime on the TopCoder online innovation contest platform. Participants competed in online "rooms" of 20 competitors, where each participant could observe detailed profiles of other competitors once the contest started. Coders were put into a rank-ordered list which the researchers split into two equally sized groups of participants with equal skill distributions by randomly assigning members of each ordered pair to one group or another (sorted and unsorted group). The sorted group competed in teams, while the unsorted participants competed individually; individuals in both groups had the same expected value for cash prizes. Roughly half of the rooms were randomised to award cash prizes to the top team (unsorted) or the 5 finishers (unsorted). Following the event, an objective score of performance for each competitor was publicly posted. Rounds of data collection Baseline data collection and method TopCoder's comprehensive database and competitors' responses to a question about competition preference. The database contains information on contests and their participants, including prizes for each contest, participants of each contest by room and team, skill ratings of contestants, number of submissions for contestants, and problem solving scores of contestants. Data collection method and data collected Evaluation Outcome variables <p>Problem Solving Score: Numerical score awarded to a solution as an assessment of overall quality, based on automated test suite. Number of Submissions: Number of solutions submitted to be compiled, tested and scored by an individual participant during the course of the experiment. Hours Worked: Number of hours worked by an individual participant during the course of the experiment.</p> Results <p>The effect of sorting on the basis of a preference for competition is large. Even after controlling for the skills of sorted individuals with the matching procedure, sorted workers still achieved roughly twice the problem-solving performance as unsorted workers. Estimated effects should be considered conservative since workers across the economy sort entirely across many dimensions such as industry, entrepreneurial firms, government bureaucracy, academia, etc. Performance differences are caused by differences in effort-level choices (behaviour). Performance differences were not related to unobserved differences in skills or productivity. Tastes for the competitive contest regime are associated with differences in underlying attitudes, preferences, and psychological and behavioural differences. The presence of cash incentives had the same magnitude of effect - a doubling of problem solving performance - as the preference for the competitive regime. Skills-based sorting is important and significant, but impacts performance by only 5%. The effect of sorting on skill was highly complementary with cash incentives, thus the presence of high-powered incentives is necessary to benefit from the effects of skill based sorting.</p> Intervention costs Not available. Cost benefit ratio Reference Boudreau, K. J., & Lakhani, K. R., 2015. 'High Incentives, Sorting on Skills -- or Just a "Taste" for Competition? Field Experimental Evidence from an Algorithm Design Contest'. Working paper. Citation for use in academic references
