Chunqiu Steven Xia and Courtney Miller.
Do These Violent Delights Have Violent Ends? Measuring the Post-Merge Fate of Agentic Code.
Currently under review, 2026.
We tracked the post-merge fate of agentic and human code across 182 repositories, following what later changes were made to it and what defects and vulnerabilities it introduced. We found that agentic code is maintained at a similar overall rate but requires significantly more corrective maintenance and introduces more security weaknesses and dependency vulnerabilities, implying that agentic tools need to be evaluated and designed not only to produce mergeable changes, but contributions that remain secure and maintainable.
Paper / Website / Code / Infographic / Details
Sadia Afroz, Courtney Miller, Tyler Menezes, Edward Gilmour, Anita Sarma, Zixuan Feng.
"AI Slop is DDoSing Open Source:" Understanding the Impact of AI-Generated Contributions on Open Source Sustainability.
Currently under review, 2026.
Drawing on practitioner accounts from Reddit, OSS mentor mailing lists, and blogs, we characterize AI-DDoS, a denial-of-service effect in which plausible but low-quality AI-generated contributions overwhelm open source communities' capacity to review them, and find the pattern in repository data too: pull request volume rose in 2025 while merge rates fell, hitting one-time contributors hardest. Through maintainer interviews and a survey of 229 practitioners, we identify 11 remediation strategies and find AI-DDoS is a sustainability trap as much as a volume problem, since communities most often endorse low-effort defenses that protect short-term review capacity while making openness hard to sustain.
Paper
Shyam Agarwal, Courtney Miller, Christian Kästner, Bogdan Vasilescu.
3100 Opinions on Code Review in an AI World: Building Causal Theory from Practitioner Discourse.
Currently under review, 2026.
We collected 38,709 engineering blog posts and forum threads where practitioners argue about code review, coded a stratified random sample of 3,100 with an LLM-assisted pipeline, and built a causal theory from them covering 26 constructs and 67 relationships. The theory's organizing claim is that review is where a coding agent's effect on a project gets decided for better or worse, and that which one you get depends on the expertise a project can draw on and how its review process is set up.
Paper / Code
Courtney Miller, Rudrajit Choudhuri, Mara Ulloa, Sankeerti Haniyur, Robert DeLine, Margaret-Anne Storey, Emerson Murphy-Hill, Christian Bird, Jenna L. Butler.
“Maybe We Need Some More Examples:” Individual and Team Drivers of Developer GenAI Tool Use.
IEEE/ACM International Conference of Software Engineering (ICSE), 2026.
Through paired interviews with 54 developers across 27 teams – one frequent and one infrequent user per team – we found key differences in how frequent and infrequent users perceive, approach working with, and respond to challenges when using GenAI tools, as well as team and organizational factors that shape their usage. Our findings imply that widespread organizational expectations for rapid productivity gains without sufficient investment in learning support creates a "Productivity Pressure Paradox," undermining the very productivity benefits that motivate adoption.
Paper / Slides / Supplementary Material / Infographic / Details
Courtney Miller*, Hao He*, Weigen Chen, Elizabeth Lin, Chenyang Yang, Bogdan Vasilescu, Christian Kästner.
Designing Abandabot: When Does Open Source Dependency Abandonment Matter?.
IEEE/ACM International Conference of Software Engineering (ICSE), 2026.
*Both authors contributed equally to this research.
We identify the contextual factors impacting dependency abandonment on a project and build a classifier to automatically predict dependency abandonment impact. Our results show that the classifier is effective at predicting whether a dependency’s abandonment would be impactful to a project, and that theory-based explanations given by the LLM are useful to developers when making judgments about the potential impactfulness of a given dependency’s abandonment.
Paper / Slides / Supplementary Material / Details
Hao He, Courtney Miller, Shyam Agarwal, Christian Kästner, Bogdan Vasilescu.
Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects.
ACM International Conference on Mining Software Repositories (MSR), 2026.
Large language models (LLMs) agents are rapidly gaining momentum in their application to software development, with some claiming a multifold productivity increase after adoption. Yet, empirical evidence is lacking around these claims. In this paper, we estimate the causal effect of adopting a popular LLM agent assistant, namely Cursor, on development velocity and software quality using difference-in-differences statistical analysis. We find that Cursor adoption leads to a signifcant, large, but transient increase in project-level development velocity, along with a signifcant and persistent increase in static analysis warnings and code complexity. Additionally, the increase in static analysis warnings and code complexity acts as a major factor causing long-term velocity slowdown. Our study carries implications for software engineering practitioners considering adopting these tools, LLM agent assistant designers, and researchers.
Paper
Mara Ulloa, Jenna L. Butler, Sankeerti Haniyur, Courtney Miller, Barrett Amos,Advait Sarkar, Margaret-Anne Storey.
Product Manager Practices for Delegating Work to Generative AI:“Accountability Must Not Be Delegated to Non-Human Actors”.
IEEE/ACM International Conference of Software Engineering -- Software Engineering in Practice Track (ICSE-SEIP), 2026.
GenAI is changing the nature of knowledge work, particularly for Product Managers (PMs) in software development teams. While much software engineering research has focused on developers’ interactions with GenAI, there is less understanding of how the work of PMs is evolving due to GenAI. To address this gap, we conducted a mixed-methods study at Microsoft: surveying 885 PMs, analyzing telemetry data for a subset of PMs (N=731), and interviewing a subset of 15 PMs. We contribute: (1) PMs’ current GenAI adoption rates, uses cases, and perceived beneits and barriers and; (2) a framework capturing how PMs assess which tasks to delegate to GenAI; (3) PMs adaptation practices for integrating GenAI into their roles and perceptions of how their role is evolving.
Paper
Courtney Miller, William Enck, Yasemin Acar, Michel Cukier, Alexandros Kapravelos, Christian Kästner, Dominik Wermke, Laurie Williams.
S3C2 Summit 2024-08: Government Secure Supply Chain Summit.
On Thursday, August 29th, 2024, three researchers from the NSF-backed Secure Software Supply Chain Center (S3C2) conducted a Secure Software Supply Chain Summit with a diverse set of 14 practitioners from 10 government agencies. The goals of the Summit were to: (1) to enable sharing between participants from different government agencies regarding practical experiences and challenges with software supply chain security; (2) to help form new collaborations; (3) to share our observations from the two summits conducted with industry in the past year; and (4) to learn about the participants' challenges to inform our future research directions.
Report / S3C2 Homepage
Courtney Miller, Mahmoud Jahanshahi, Audris Mockus, Bogdan Vasilescu, Christian Kästner.
Understanding the Response to Open-Source Dependency Abandonment in the npm Ecosystem.
IEEE/ACM International Conference of Software Engineering (ICSE), 2025.
We perform a large-scale quantitative analysis of all widely-used npm packages and find that abandonment is common among them, that abandonment exposes many projects which often do not respond, that responses correlate with other dependency management practices, and that removal is significantly faster when a package’s end-of-life status is explicitly stated. We end with recommendations to both researchers and practitioners who are facing dependency abandonment or are sunsetting packages, such as opportunities for low-effort transparency mechanisms to help exposed projects make better, more informed decisions.
Paper / Slides / Supplementary Material / Infographic / Details
Giacomo Benedetti, Oreofe Solarin, Courtney Miller, Greg Tystahl, William Enck, Christian Kästner, Alexandros Kapravelos, Alessio Merlo, Luca Verderame.
An Empirical Study on Reproducible Packaging in Open-Source Ecosystems.
IEEE/ACM International Conference of Software Engineering (ICSE), 2025.
We quantitatively studied reproducible builds in a novel context: reusable components distributed as packages in six popular software ecosystems (npm, Maven, PyPI, Go, RubyGems, and Cargo). We found that rates of reproducible builds vary widely across ecosystems and that infrastructure configurations and patching of build tools can help achieve very high rates of reproducible builds in all studied ecosystems.
Paper / Supplementary Material
Lina Boughton, Courtney Miller, Yasemin Acar, Dominik Wermke, and Christian Kästner.
Decomposing and Measuring Trust in Open-Source Software Supply Chains.
IEEE/ACM International Conference of Software Engineering -- New Ideas Track (ICSE-NIER), 2024.
Trust is integral for the successful and secure functioning of software supply chains, making it important to measure the state and evolution of trust in open source communities. However, existing security and supply chain research often studies the concept of trust without a clear definition and relies on obvious and easily available signals like GitHub stars without deeper grounding. In this paper, we explore how to measure trust in open source supply chains with the goal of developing robust measures for trust based on the behaviors of developers in the community. To this end, we contribute a process for decomposing trust in a complex large-scale system into key trust relationships, systematically identifying behavior-based indicators for the components of trust for a given relationship, and in turn operationalizing data-driven metrics for those indicators, allowing for the wide-scale measurement of trust in practice.
Paper / Supplementary Material / DOI
Courtney Miller, Christian Kästner, Bogdan Vasilescu. "We Feel Like We're Winging It:" A Study on Navigating Open-Source Dependency Abandonment. ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2023.
We conduct semi-structured, in-depth interviews with 33 developers who have experienced open-source dependency abandonment and perform iterative qualitative content analysis to collect, curate, and contextualize the experiences and practices of developers who have dealt with open-source dependency abandonment.
Paper / Slides / Presentation Video / Supplementary Material / Details
Courtney Miller, Sophie Cohen, Daniel Klug, Bogdan Vasilescu, Christian Kästner.
"Did You Miss My Comment or What?" Understanding Toxicity in Open Source Discussions.
IEEE/ACM International Conference of Software Engineering (ICSE), 2022.
We take a first stride at understanding the characteristics of open source toxicity to better inform future work on designing effective intervention and detection methods. We curate and qualitatively analyze a sample of 100 toxic GitHub issue discussions to gain an understanding of the characteristics of open-source toxicity.
Paper / Slides / Presentation Video / Infographic / Details
Kimberly Truong, Courtney Miller, Bogdan Vasilescu, and Christian Kästner.
The Unsolvable Problem or the Unheard Answer? A Dataset of 24,669 Open-Source Software Conference Talks.
IEEE/ACM International Conference on Mining Software Repositories (MSR), 2022.
Talks at practitioner-focused open-source software conferences are a valuable source of information for software engineering researchers. They provide a pulse of the community and are valuable source material for grey literature analysis. We curated a dataset of 24,669 talks from 87 open-source conferences between 2010 and 2021. We stored all relevant metadata from these conferences and provide scripts to collect the transcripts. We believe this data is useful for answering many kinds of questions, such as: What are the important/highly discussed topics within practitioner communities? How do practitioners interact? And how do they present themselves to the public? We demonstrate the usefulness of this data by reporting our findings from two small studies: a topic model analysis providing an overview of open-source community dynamics since 2011 and a qualitative analysis of a smaller community-oriented sample within our dataset to gain a better understanding of why contributors leave open-source software.
Paper / DOI
Courtney Miller, Paige Rodeghero, Margaret-Anne Storey, Denae Ford, Thomas Zimmermann.
"How Was Your Weekend?" Software Development Teams Working From Home During COVID-19 .
IEEE/ACM International Conference of Software Engineering (ICSE), 2021.
We investigate how development team collaboration, communication, and productivity have been impacted by the rapid shift to remote work during the COVID-19 pandemic. We perform two surveys at a large software company to identify the challenges relating to inter and intra team collaboration and communication and use statistical modeling to quantify how those challenges impact changes in self-reported productivity levels.
Paper / Slides / Presentation Video / Supplementary Material / Details
Courtney Miller, David Gray Widder, Christian Kästner, Bogdan Vasilescu.
Why Do People Give Up FLOSSing? A Study of Contributor Disengagement in Open Source.
IFIP International Conference on Open Source Systems (OSS), 2019.
We conduct a mixed-methods empirical study, combining surveys and survival modeling, to identify the reasons and predictive factors behind established open source contributor project disengagement.
Paper / Slides / Details
