AI coding tools can accelerate development, but faster implementation does not always mean faster or more reliable software delivery. Recent research raises important questions about testing, debugging and the skills engineers need.
Cover illustration generated for this article. It is an AI-generated scene, not a photograph of a real incident, and the on-screen text and timestamps are synthetic.
A developer asks an AI coding agent to improve the retry logic in a payment-processing service.
The agent examines the code, updates the implementation, generates tests and reports that everything passes. The developer reviews the changes and approves the release.
A few days later, an intermittent network failure causes some payments to be processed twice.
The code works as written. The problem is an assumption about how another system behaves.
This is a hypothetical example, but it illustrates an important engineering distinction: code can pass its tests and still cause problems when interacting with other systems.
As AI coding tools become more capable, understanding those interactions matters as much as generating the implementation.
What recent research tells us
In September 2026, debugging technology company Undo published a survey of 300 senior engineering leaders working in complex software environments.
Among the respondents, 81% reported that their organizations had experienced a production incident or outage associated with AI coding tools during the previous six months. Another 79% reported no improvement in release-cycle speed despite increased code production.
These figures deserve attention, but they also require context. The research was vendor-commissioned and based on self-reported experiences. It does not establish that AI caused every reported incident or that the findings represent the entire software industry.
Independent studies provide a more varied picture.
| Research | Key finding |
|---|---|
| Undo, 2026 | 81% of surveyed leaders reported AI-coding-related production incidents. |
| Google DORA, 2025 | AI adoption was associated with higher delivery throughput, while software delivery stability remained a concern. |
| METR, 2025 | Experienced developers took 19% longer on selected tasks when using the AI tools available during the experiment. |
| METR, February 2026 | Newer tools appeared more promising, but measurement limitations prevented a reliable updated productivity estimate. |
| GitHub Copilot research | A controlled experiment found faster completion of a defined programming task with AI assistance. |
The studies measure different things under different conditions. They should not be treated as directly comparable results.
What they suggest is that the benefits of AI coding tools depend on the task, the software environment and the engineering practices around them.
Generating code faster is useful. Whether it improves software delivery is a separate question.
Why passing tests may not be enough
Return to the payment-processing example.
Suppose an application sends a payment request to an external provider. The provider processes the payment successfully, but the network connection fails before the application receives confirmation.
The application cannot immediately determine whether the payment succeeded.
If it retries without appropriate safeguards, the customer could be charged again.
One established solution is an idempotency key, a unique identifier that allows the payment provider to recognize repeated attempts involving the same transaction.
But consider what happens if the AI-generated retry logic creates a new identifier for every attempt.
A unit test checking whether the application retries after a timeout might pass. Yet the implementation could still allow duplicate payments.
The test confirms the retry mechanism, not the complete transaction behavior.
An experienced engineer would examine the requirement more broadly: a temporary network failure must not result in the same logical transaction being processed twice.
This is an example of a system invariant, a condition that must remain true regardless of retries, failures or concurrent operations.
AI tools can help implement and test these conditions. The difficulty arises when important requirements are absent from the prompt, documentation or available code context.
Accessible text alternative for this figure
Three steps in a row. Step 1, initial request: a 50 dollar payment request with key order-145, and the provider charges successfully. Step 2, response lost: a network timeout, so the application cannot tell whether the payment succeeded. Step 3, risky retry: a new key order-146, which the provider may treat as another payment, resulting in a duplicate.
Below the steps, the safer invariant: one logical order uses one consistent idempotency key. Retry with key order-145 rather than creating a new key per attempt, and test the provider-facing behavior, not just whether the retry code runs. Provider behavior and API contracts vary, so the diagram illustrates the engineering principle only.
The importance of understanding production behavior
Some software defects cannot be identified by examining individual functions.
They appear only when multiple services interact, requests arrive concurrently or unexpected failures occur.
Consider an application where two services update the same database record.
Each service may behave correctly in isolation, but their operations could conflict when executed simultaneously.
Diagnosing such failures requires more than reading the source code.
Engineers may need application logs, distributed traces, database records and runtime metrics to reconstruct the sequence of events.
These tools are collectively part of observability: the ability to understand what a system is doing by examining the information it produces.
AI can assist with this investigation by analyzing logs, tracing requests and suggesting possible causes.
However, a plausible AI-generated explanation is not necessarily the correct root cause. Engineers must compare it with actual runtime evidence.
This is where experience with system architecture and debugging becomes valuable.
What changes in code review?
AI-assisted development can increase the amount of code produced in a short period.
A developer may submit an implementation containing several modified services, database changes and automatically generated tests.
Reviewing such changes requires more than checking syntax or readability.
The reviewer needs to understand the consequences.
Does the change preserve existing business rules? Could it introduce a security weakness? What happens when another service is unavailable? Will existing data remain valid?
The level of review should reflect the risk.
A documentation update does not require the same verification as a change to payment processing, authentication or database transactions.
For critical changes, engineering teams should require appropriate integration tests, regression checks and operational safeguards regardless of whether the code was written manually or generated by AI.
The objective is not to create additional approval processes simply because AI was involved.
It is to ensure that faster implementation does not bypass existing engineering responsibilities.
Which engineering skills become more important?
AI coding tools are changing how developers spend their time.
Tasks that once required substantial manual coding may now begin with a generated implementation.
That creates opportunities for productivity, but it also changes how engineers develop experience.
A junior developer who accepts generated code without examining it may complete an assignment while learning relatively little about the system.
The same developer could use AI-generated code to investigate database transactions, API security, error handling and application architecture.
The difference is how the tool is used.
Several established engineering skills remain particularly important.
| Skill | Why it matters |
|---|---|
| System architecture | Understanding how changes affect connected components |
| Code comprehension | Evaluating generated implementations and their assumptions |
| Testing | Identifying conditions that must remain true under failures |
| Debugging and observability | Finding causes of failures in running systems |
| Security engineering | Recognizing authorization, dependency and data-protection risks |
| Domain knowledge | Connecting implementation details to business requirements |
These skills are not replacing programming knowledge.
Engineers still need to understand code well enough to recognize incorrect behavior and unsafe implementations.
AI may reduce the time spent manually writing certain functions, but it does not eliminate the need for technical judgment.
For engineering managers, this also raises a training question: if AI performs much of the straightforward implementation work, how will less-experienced engineers gain the practical experience needed to handle complex systems?
That deserves attention alongside productivity targets.
Measuring productivity beyond generated code
An engineering team can produce more code without delivering more value.
Additional implementation may lead to longer reviews, unexpected regressions or greater debugging effort.
Organizations should therefore evaluate AI-assisted development through the complete delivery process.
Useful measurements include:
- Time required to deliver a change.
- Frequency of successful deployments.
- Production failures associated with changes.
- Time required to diagnose and recover from incidents.
- Defects discovered after release.
These measures need to be interpreted together.
A team deploying more frequently is not necessarily improving if production failures are also increasing.
Likewise, a reduction in manually written code does not automatically indicate reduced engineering effort if additional time is needed for verification.
The objective should be reliable delivery, not maximizing the quantity of generated code.
What this means for software engineering careers
AI coding tools are becoming part of normal software development.
Their capabilities will continue to change, and their benefits will differ across applications and organizations.
The research available today does not support a universal conclusion that AI coding makes engineering teams faster or less reliable.
It does highlight the importance of looking beyond implementation speed.
Software engineers increasingly need to evaluate generated code, understand its interactions with existing systems and investigate behavior that only becomes visible during operation.
For students and early-career developers, learning to use AI tools is valuable. So is developing the programming, debugging and architectural knowledge needed to recognize when those tools are wrong.
For experienced engineers, the opportunity is to spend less time on routine implementation while applying their knowledge to more complex problems.
AI may change how much code engineers write. It does not remove the need to understand the software they deliver.
Sources and further reading
Editorial source cutoff: 14 October 2026. Each source was opened and checked on 14 October 2026.
- Undo: Overcoming the Limitations of Coding Agents in Complex Software Systems. September 2026. Vendor-sponsored survey of engineering leaders.
- Google DORA: State of AI-assisted Software Development 2025. 2025. AI adoption and software delivery outcomes.
- METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. July 2025. Controlled experiment.
- METR: We are Changing our Developer Productivity Experiment Design. February 2026. Follow-up findings and measurement limitations.
- GitHub: Research: quantifying GitHub Copilot’s impact on developer productivity and happiness. Controlled productivity experiment.