Human-AI Collaboration in Software Teams: Evaluating Productivity, Quality, and Knowledge Transfer with Agentic and LLM-Based Tools

Abstract

The ascendency of large language models (LLMs) and the nascent age of agentic AI systems are changing how software teams work, create, and maintain code. These are tools that have evolved from static code-completion helpers to mission-critical software development cycle tools that affect productivity, quality of code, and how knowledge is created and transferred among teams. However, despite being widely deployed, fine-grained effects on socio-technical systems are understudied, especially at the team level, where human-human interactions meet with AI augmentation. This paper offers a systematic review of human-AI collaboration in software teams, focusing on productivity, quality, and knowledge transfer by introducing a three-dimensional framework for viewing this phenomenon. Unlike previous works, which mainly concentrate on individual developer productivity or task-level efficiency, this work shines the spotlight on team-oriented processes and organizational rules or governance forms that decide whether AI adoption brings sustained value or ominous risk. The framework is informed by evidence based on randomized controlled trials, field deployments, and benchmark evaluations. In the control studies (e.g., GitHub Copilot RCTs), we observe that time can be saved up to 55–56% for bounded coding tasks, compared to enterprise case studies, where reported improvement in developer happiness and reduction in mental energy consumption coincide. Nevertheless, repository-level benchmarks such as SWE-bench indicate that LLMs/agents still have room to catch up on complex multi-file changes and subtle bug-fixes, supporting the requirement of a human-in-the-loop governance and robust validation pipelines. By the same token, research on AI-augmented code review and human-AI pair programming reveals efficiency gains as well as trade-offs that are far from trivial: reviews are faster and junior engineers may be onboarded more easily, but critical thinking skills atrophy and dialogic inspection often weakens, inviting knowledge dilution and undiagnosed design mistakes. Methodologically, this paper is based on a quasi-experimental mixed-methods study design reproducible in enterprise software development environments. The solution leverages sprint-wise cohort contrast (baseline, LLM-supported, and agentic), backlog-aligned task batteries, as well as multi-modal instrumentation of developer efforts. You’re not judged by cycle time alone, but by the ratio of rework, velocity through your backlog, and cost of context switching. Quality is assessed by pre-merge signals (test pass rates, static analysis hits, review loops) and post-release results (defect density, incident frequency, and rollback rates). Knowledge transfer is modelled by analysing reasons for PRs, design decision logs, pair-programming dialogues, and file-ownership distributions as indicators of the organisational memory. Data triangulation contributes to validity, with quantitative telemetry complemented by qualitative interviews about trust, usability, and team dynamics. The utilization of the framework over typical examples validates several intuitive insights. First, AI copilots are consistently more productive when it comes to scaffolding, test generation, and documentation tasks, except for where value boundaries are poorly defined or on architectural work. Second, quality is enhanced when AI output is gated by disciplined engineering practices — e.g., tests, linters, and structured reviews but degraded when excessive trust causes unfettered acceptance of plausible-but-wrong code. Third, the nature of knowledge transfer dynamics changes rather than disappears: humans and machines can be trained together quickly at a surface level but not so much at dialogic deep learning unless such interactions are assisted by protocols that enforce rationale capture, architecture reviews, and peer-to-peer explainability. Crucially, comfort gains and cognitive effort reductions are tempered by survey data showing declining trust in accuracy, illustrating a paradox of enthusiastic skepticism among developers. The discussion section re-mediates the results into an adopter playbook for the industry, and provides maturity models for blending LLM and agentic tools. Main takeaways are progressive autonomy between (suggestion-only to constrained edits to supervised multi-file changes), policy guardrails (security scanning, coverage thresholds, auditability), and socio-technical rituals (review-by-explanation, decision records, AI-generated rationale validated by humans). Through a disciplined, workflow-oriented framing of human-AI collaboration as not replacement but augmentation, this paper posits the LLM and agentic toolset as levers to drive productivity efficiency and quality enhancement at knowledge transfer rates that minimize loss risk to organizations.