Falling Asleep at the Wheel: Human/AI Collaboration in a Field Experiment on HR Recruiters

I investigate how firms should design human/AI collaboration to ensure human workers remain engaged in their activities. I developed a formal model that explores the tension between AI quality and human effort. As AI quality increases, humans have fewer incentives to exert effort and remain attentive, allowing the AI to substitute, rather than augment their performance. Thus, high-performingalgorithms may do worse than lower-performing ones in maximizing combined output. I then test these predictions using a pre-registered field experiment where I hired 181 professional recruiters to review 44 resumes. Iselectedarandomsubsetofscreenersto receive algorithmic recommendations about job candidates, and randomly varied the quality of the AI predictions they received. I found that subjects with higher quality AI wereless accurate in their assessments of job applications than subjects with lower quality AI. On average, recruiters receiving lower quality AI exerted more effort and spent more time evaluating the resumes, and were less likely to automatically select the AI-recommended candidate. The recruiters collaborating with low-quality AI learned to interact better with their assigned AI and improved their performance. Crucially, these effects were driven by more experienced recruiters. Overall, the results show that maximizing human/AI performance may require lower quality AI, depending on the effort, learning, and skillset of the humans involved.