Part 2: AI Adoption Is a Capability, Not a Tool: What a Before-and-After Look at Our Own Developers Showed
We compared each of our developers' delivery records before and after AI became part of everyday work: same people, same kind of work, only the environment changed. Part 2 of our AI productivity series explores what that shift revealed, and what it means for teams investing in AI.
Giving a team access to AI is not the same as adopting it. In our latest round of delivery analysis, we did something we couldn't do in our first study back in November 2025: we compared each developer's delivery record before broad AI availability against their own record after. Same people, same estimating habits, same kind of work, with the caveat that only the environment changed.
This is Part 2 of our three-part AI productivity series. Part 1 covered how we measure AI use honestly, the ground-truth foundation everything here rests on. In this part, we compare each developer's delivery record before broad AI availability against their own record after, to see what adoption changed. Stay tuned for Part 3, as it closes the series with why fixed-price delivery gets safer with AI.
Why a before-and-after on the same person is the strongest signal
Aggregate averages hide as much as they reveal. Comparing one group of developers to another introduces a dozen confounds, different work, different seniority, different clients. The cleanest comparison is a person against their own past, so we hold the individual constant and let only the era change.
A word on what we're measuring. Estimating software work is hard, and real delivery always lands in a spread around the estimate; some tasks come in under, many land on the money, and some run over. That spread is normal and expected, and not a sign that anything is wrong. What matters commercially is the smaller number of tasks that run well over estimate, because those are what distort budgets, timelines, and client confidence. So, when we look at a developer's delivery, we are watching that spread not holding anyone to a standard of never exceeding an estimate. On top of that, we also look at particularly how often work runs well over here.
We had enough history to do exactly that for a large share of our long-tenured developers, which are people with a substantial track record both before and after AI became part of everyday delivery. For each one we compared how their work sat against an estimate before and after. We put special emphasis on how often it ran over.
Most developers' work landed closer to estimate
The majority of developers with enough history in both periods saw fewer of their tasks run overestimate in the AI era than in their own pre-AI baseline, and more land on or under budget. Not a handful of champions dragging an average up, but a broad majority, each measured against themselves.
And the shift among the most committed users was not marginal. Several roughly halved how often their work ran well overestimate. The developers who moved the other way were, with a few exceptions, those with very little recent history rather than genuine regressions. This is also where a single hard task swings the picture. Estimation is uncertain about work, and some movement in both directions is exactly what you would expect.
Each bar is one developer's change in how often their work ran overestimate, measured against their own pre-AI baseline. Most ran over less often than before. This matters because it rules out the most common objection to AI productivity claims. This isn't a story about new hires being better, or easy work being cherry-picked. It's the same people, doing their kind of work, delivering more reliably than they used to.
The effect scales with depth of use
When we looked at how heavily AI was used on a task, a dose-response pattern emerged. Light AI involvement moved the needle a little. Heavy involvement, meaning AI as the primary tool throughout the work and not an occasional lookup moved it a lot, with the costly tail of large overruns on those tasks falling close to zero.
That points to something important about adoption. The value isn't unlocked by occasional use. It compounds as AI becomes the way the work is done. The leading edge of our team isn't using AI to assist a traditional workflow. They've built a new workflow around it.
Work runs overestimate less often as AI use deepens. The heaviest use pushes the costly tail close to zero.
What full adoption actually looks like
The most striking case in our data was a developer whose aggregate numbers looked unremarkable, even poor, until we read them in sequence rather than in summary.
Historically, this person was prone to large overruns. Some early work ran multiples over estimate. Then, over a stretch of recent months, every single delivered task came in on or under budget. A long, unbroken run of reliable delivery from someone whose history said the opposite. The turning point mapped precisely to when AI became a consistent, central part of how they worked. And this same developer is now among our very heaviest AI users.
The aggregate average buried this completely, because it averaged a transformed present with a struggling past. Read as a trajectory, it's one of the clearest before-and-after arcs we have. It’s someone who was a delivery risk became one of our most reliable people, and the change tracks their adoption depth almost exactly.
The lesson we took from it is the theme of this post. The developers showing transformed outcomes are the ones who committed to AI as a primary way of working, and not the ones who were simply given access to it.
Adoption is a coaching problem, not a procurement problem
This reframes how we think about AI investment, and it's the part most relevant to other organisations.
Buying licences is easy and it's where most AI spend goes. But our data says the return doesn't come from access, but from depth of adoption, and depth is uneven across any team. Some people build a new workflow around AI quickly. Others keep it at arm's length and see modest gains. The gap between them is not capability of the tool, but how the person has integrated it.
That makes AI adoption a capability-building exercise, like training, coaching, sharing what works, pairing people with effective patterns to be far more than a tooling exercise. The organisations that treat it as the former will pull ahead of the ones that treat it as the latter, even on identical tools.
AI Adoption: Key Findings
Finding 1: Comparing developers to their own past is the cleanest signal. Holding the person constant and changing only the era removes the confounds that muddy group-versus-group comparisons.
Finding 2: A broad majority delivered more predictably. Most developers with enough history in both periods saw fewer of their tasks run over estimate in the AI era than they used to, with more work landing on or under budget which is also measured against themselves.
Finding 3: The effect is dose-dependent. Light AI use helped a little while heavy use, where AI was the primary tool throughout, helped a lot with tail risk on those tasks falling close to zero.
Finding 4: Full adoption can be transformative at the individual level. Our clearest case is a developer who went from frequent large overruns to a long unbroken run of on-or-under-budget delivery, tracking exactly with their depth of AI use.
Finding 5: Adoption is a capability, not a purchase. The return comes from depth of use, not access. That makes training, coaching, and pattern-sharing the high-leverage investment. It’s not licences alone.
AI Adoption FAQs
Q: Isn't "developers improved" just because the work got easier or the team changed?
It’s the exact reason why we compared each developer to their own past rather than to other people. When you hold the individual constant, like same person, same type of work, same estimating habits, changes in outcome can be attributed to the environment. Run that analysis on your own teams, and you'll know whether AI is genuinely moving the needle or whether something else is driving the numbers.
Q: On fixed-price work, who absorbs an overrun? Is it you or the client?
We do, so you need to protect your margin by targeting AI adoption at the work that historically runs over. On a fixed-price engagement, overruns come out of our margin, and the client pays the agreed price regardless. That makes delivery variance a direct cost to manage. You will need to identify the task types and developers where overruns are most frequent, focus adoption effort there first, and you'll see the biggest commercial return fastest.
Q: Why should a client care about your overrun rate, then?
Two reasons. First, a delivery partner running tight delivery variance is under less margin pressure. This means healthier relationships, fewer difficult commercial conversations, and since more work lands on or under estimate, working software in your hands sooner.
Second, as a client, look for delivery partners whose overrun data shows a downward trend. This is a signal that their fixed-price commitments are ones they can actually keep. The price is the price, delivered with fewer surprises and, often, earlier than expected.
Q: What does "dose-dependent" mean here?
It means to make AI your primary tool throughout your highest-risk tasks, not just an occasional assist. The data shows a clear relationship here that the more intensively AI was used on a task, the better the delivery outcome. Occasional use produces small gains, so you need to commit to AI as the main tool throughout to produce the largest improvement. This comes with extreme overruns becoming rare even on hard work, and that’s why depth of use is what unlocks the protection.
Q: Does giving everyone AI licences deliver these results?
Not on its own. You will need to pair every license rollout with a structured capability programme. Access is the starting point, not the finish line. Our data shows that teams with identical tools produce very different outcomes depending on how deeply individuals have developed their AI workflows. The license opens the door, but it’s the training, coaching, and shared practice on your end that get people through it.
Q: So how do you actually build adoption?
Run it like a capability programme. It needs structured onboarding, regular coaching, and active sharing of the workflows and patterns that actually move the numbers. The practices that produce results need to spread across the whole team, not stay with a few people who figured it out themselves. Build systems for that spread, and you’ll see how adoption compounds. Also, the early gains accelerate rather than plateau.
Q: Can these gains be expected immediately?
Short answer is no, but you can plan for a capability curve and invest in shortening it. The deepest improvements came from sustained, committed use and not first contact with the tool. Set an adoption target, measure delivery outcomes at each milestone, and use the data to identify where the curve is steepening and where it needs more support. Organisations that invest in shortening the curve see the gains earlier and with less attrition along the way.
Building AI Capability in Your Team
The hard part of AI adoption isn't the tooling, it's turning access into a way of working that actually changes outcomes, consistently and across a whole team. We've measured that journey inside our own delivery and learned where the leverage is.
This was Part 2 of 3 of the AI productivity report series. If you missed it, Part 1 covers how we measure AI use honestly, the foundation this analysis is built on. Coming up, Part 3 looks at why fixed-price delivery gets safer with AI.
Ready to See AI in Action Within Your Team?
We help organisations build the same AI capability in theirs. So if you want to see how it looks like in your team, book a free discovery call today. No prep needed, just a conversation about where you are and where the gains are.
