This paper discusses several challenges in designing field experiments to better understand how organizational and institutional design shapes innovation outcomes and the production of knowledge. We proceed to describe the field experimental research program carried out by our Crowd Innovation Laboratory at Harvard University to clarify how we have attempted to address these research design challenges. This program has simultaneously solved important practical innovation problems for partner organizations, like NASA and Harvard Medical School, while contributing research advances, particularly in relation to innovation contests and tournaments.
Various. Among the most prominent variables: venture survival, venture growth, venture profitability, financial well-being of the clients, health of clients, education of the clients' children, and empowerment of female clients.
Challenges with field experiments applied to economics of innovation include: 1) the nature of the knowledge production function, 2) unit of analysis, replication and sample size, 3) selection versus treatment effects, 4) designing treatments and counterfactuals, 5) representativeness, validity and precision. The Crowd Innovation Laboratory's work has shown that innovation contests can be used to solve computational problems and has demonstrated significant gains in cost effectiveness, speed to solution and quality. Development of experiments focused around the generation and evaluation of scientific research grant proposals has also advanced knowledge in economics of innovation related to how researchers compete and collaborate to win grants. The policy challenge now is to understand how innovation contest can become a routine part of federal procurement for technology.