Research in Progress in Human-AI Collaboration

 ODC Webinar

Research in Progress in Human-AI Collaboration

Thursday, December 12, 2024 at 3PM CET

In this ODC "Research in Progress" (RiP) webinar, organized and moderated by Arianna Marchetti (London Business School), two PhD students - Albert Roh (USC Marshall School of Business) and Matthias Tröbinger (ESSEC) - working on research human-AI collaboration, present their working papers on “How organizational factors beyond technology shape human-AI collaboration” and “How experience moderates the impact of generative AI ideas on the research process” (each about 20 mins). The presentations will be followed by a commentary by Felipe Csaszar (Ross School of Business, University of Michigan). The webinar will end with feedback/comments from the audience. Please see below for the papers’ abstracts and the participants’ bios.

Arianna Marchetti [Organizer and Moderator]

Arianna is an Assistant Professor at the Department of Strategy and Entrepreneurship at London Business School. Her research lies at the intersection of strategy and organization, taking a keen interest in the interplay between formal organization design and organizational culture and its implications for coordination and corporate performance. Another of her interests is machine learning and AI, how they can optimally collaborate with humans in organizations, and their impact on research and society at large. Before joining London Business School, Arianna obtained her Ph.D. in Management from INSEAD.

Albert Roh [Presenter]

Presentation Title: Beyond Technology: How Organizations Shape Human-AI Collaboration.

Abstract: Artificial intelligence (AI) technologies are now widely utilized across various sectors of business and society. While research has focused on comparing AI’s strengths and weaknesses to those of humans, this competition-centric view overlooks the potential of human-AI collaboration. Understanding and fostering this collaboration is crucial not only for maximizing AI’s impact but also for redefining human roles. Our research emphasizes the role of organizations in shaping human-AI collaboration, by developing a conceptual framework that examines how various organizational factors critically influence the outcomes of these interactions. We differentiate between two collaboration models: the “division of labor” and “ensemble” approaches. In the former, AI and humans perform different tasks, while in the latter, they perform the same tasks. We highlight the judgment calls that organizations and their decision-makers need to make when aggregating AI- and human-generated predictions in the ensemble approach. Various organizational elements can lead to deviations from optimal aggregation. We argue that the effectiveness of human-AI collaboration depends not just on the technology’s capabilities but also on the organizational structures and dynamics that shape its use. This study highlights that the sustained competitive advantage of AI adoption resides in an organization’s ability to achieve higher levels of human-AI collaboration.

Bio: Albert Roh is a third-year Ph.D. student in Management and Organization at the USC Marshall School of Business. His research focuses on the external factors driving innovation in firms and firms’ responses to those changes. In this context, he studies how government regulations affect innovation strategies, how technological developments shape modes of collaboration and design of incentive schemes, with a special focus on AI.

Matthias Tröbinger [Presenter]

Presentation Title: How Experience Moderates The Impact Of Generative AI Ideas On The Research Process.

Abstract: At the heart of scientific discovery are expert scientists who identify research ideas worthy of inquiry. While generative artificial intelligence (AI) technologies - large language models, in particular - have been found to outperform humans in some tasks, their impact on generating research ideas, one of the most fundamental tasks in science, remains underexplored. We investigate how the use of generative AI affects the generation of scientific ideas in the form of research proposals and scientists’ attitudes toward integrating generative AI ideas into their research process. Through a randomized online experiment with 310 scientists across research disciplines, we study how generative AI ideas affect researchers’ self-evaluation of their proposals, research agenda, and other attitudes. We do not find any average effect on their assessment of the proposals’ novelty or feasibility. However, research experience is an important moderator: experience negatively moderates the effect of generative AI on the perceived novelty and impact of the proposal, as well as views on their own research agendas. Further analyses suggest that less experienced researchers expressed acceptance of AI, arising primarily from views that new lines of thinking were triggered, but also from validation of existing ideas. More experienced researchers expressed aversion arising primarily from discounting outside ideas, but also from hesitation towards technology, and a perceived challenge to one’s identity. Our findings contribute to the innovation literature by offering initial insights into generative AI’s role in the research idea generation process, and to the growing literature on generative AI’s role in complementing human tasks.

Bio: Matthias is a PhD candidate in Management at ESSEC and a visiting PhD at the Institute of Responsible Innovation at the University of St. Gallen. His research investigates questions for fostering large-scale collaborative innovation. Specifically, he studies interventions that solution seekers can use to harness support from distributed participants and improve collective decision-making. His studies span various decentralized contexts, including crowdfunding and online deliberation. Methodologically, Matthias combines multi-case studies and online experiments with text analysis. He holds an MBA and a BSc in Economics.

Felipe Csaszar [Discussant]

Felipe is a professor of strategy and chair of the Strategy Department at the University of Michigan’s Ross School of Business. His research examines how decision structures influence key organizational outcomes, such as innovation, financial performance, and social impact. He focuses on three types of decision structures: managers’ cognitive frameworks, organizational decision-making processes, and the emerging role of artificial intelligence in decision-making.  His work provides insight into how firms can enhance their strategic capabilities and successfully navigate increasingly complex business landscapes.  More broadly, he is interested in combining formal modeling and empirical approaches to understand how firms can make better strategic decisions. His work has been published in top academic journals, including Management Science, Organization Science, Strategy Science, and the Strategic Management Journal.  He currently serves as Senior Editor for both Strategy Science and Management Science, and he previously held the same role at Organization Science.  He is also a co-editor of the upcoming Handbook of AI and Strategy. Additionally, he has served as president of the Strategy Science division of INFORMS, program chair of the SMS Behavioral Strategy division, and as a member of the executive committee of the Academy of Management Strategy division. Prior to joining the Ross School of Business, he was an assistant professor of strategy at INSEAD. He received his PhD in strategy from The Wharton School, University of Pennsylvania. Before pursuing his PhD, he was head of research at an asset management firm and CEO of an Internet startup.

Hope you will be able to join us!