The AI Revolution in Software Development: What's Real vs. Hype

Most arguments about AI and software engineering are unfalsifiable because the two sides are measuring different things. One side points at a demo where a model builds a working application from a paragraph of English. The other points at a fourteen-year-old billing system where a one-line change requires knowing which of six downstream consumers still reads a column that was renamed in 2019 and is aliased in a view for compatibility. Both are telling the truth. They are describing different parts of the job.

What follows is an attempt to draw the line precisely: what has genuinely changed in day-to-day engineering, what has not changed at all, and — the part that gets the least attention — what has merely moved somewhere else. We have been building software from Bucharest for twenty years and running agent-assisted delivery on real estates for the last two. The pattern has been consistent enough to be worth writing down.

The one thing that actually changed

Reduce it to a sentence: producing plausible code became close to free. Not correct code — plausible code. Syntactically valid, idiomatic for the framework, consistent with the surrounding style, and wrong in ways that require reading to detect.

That reads like a criticism. It is not. For a large share of engineering work, plausible covers most of the distance, and closing the remainder by review is genuinely faster than writing from scratch. The point is narrower and more consequential: the economics of the pipeline changed at exactly one station. Everything that used to be cheap relative to typing — deciding what to build, reviewing, testing, understanding the system you inherited, being the person who fixes it at 3am — is now the expensive part by comparison. Making one station ten times faster does not make the pipeline ten times faster. It makes the next station the bottleneck, usually loudly.

So the useful question is never whether a model can do something. It is what the work costs after verification. Answered honestly, that question sorts the entire field into things that got cheaper, things that did not, and things that quietly changed address.

Where the gains are real

Five categories of work have genuinely moved, and they have moved a lot. They share a property: the output is mechanically checkable, or the alternative was nothing at all.

• Bulk generation inside a specified boundary. CRUD surfaces, API clients from a schema, validation layers, adapters between two typed interfaces, the fourth implementation of a port you already designed. Work where the shape is known and the tedium is real.
• Test authoring, especially characterization tests. Pointing an agent at an untested legacy module and asking it to pin the current behavior is the highest-return use we have found. It is also the most underused, because it does not demo well.
• Migration mechanics. Framework major versions, deprecated APIs, class components to hooks, one date library to another, a SQL dialect change, a protobuf bump across forty call sites. Bounded, repetitive, verifiable, and historically the work nobody could justify funding.
• The mechanical layer of code review. Missing error paths, unawaited promises, resource leaks, unbounded queries, inconsistent null handling. An agent catches these more reliably than a tired human on the eleventh file of a large diff. It has nothing useful to say about whether the change should exist.
• Documentation of what code does. A 600-line stored procedure from 2011 with no comments becomes a readable first draft in a minute. That draft is worth considerably more than the nothing it replaces.

Rough numbers, hedged because they are ours rather than universal. On greenfield work with clear specifications and a fast test suite, we see cycle-time reductions in the 30-50% range, and they hold. On a mature estate — fifteen years of accumulated decisions, partial coverage, environments that do not match production — the honest figure is closer to 10-20%, and nearly all of it lands in tests, migrations and documentation rather than in feature delivery. Anyone quoting a flat multiplier across both situations has not worked on the second one recently.

Migration deserves its own note, because it is where the economics are least ambiguous. A version upgrade across a few hundred files is bounded, mechanically verifiable, and was previously priced at weeks of senior time that no roadmap would approve. The price fell far enough to make deferred upgrades tractable again. We have watched a team clear four years of postponed dependency work in a fortnight, with the test suite as the referee. That is a change in what is affordable, not a productivity anecdote.

The testing gain comes with a warning that routinely gets skipped. Characterization tests pin behavior, including behavior that is wrong. That is precisely what you want before a refactor and precisely what you must not call a correctness suite. Generated tests describe the code; they know nothing about your intent. A green suite that asserts the bug is worse than no suite at all, because it is trusted.

Where the gains are not real yet

Four things have not moved, and it is worth being specific about why. The reasons are structural rather than a matter of waiting for the next model release.

Autonomous ownership

Agents complete tasks. Ownership is a different object: the standing commitment to notice something is wrong before a customer does, to hold the reasons the system is shaped the way it is, and to decide at 2am whether to roll back or fix forward. That commitment runs on a timescale no current system holds state across, and it has stakes no current system carries. Delegating a task is fine and often correct. There is nobody to delegate ownership to.

Architectural judgment

Architecture is mostly deciding which future you are willing to pay for. Service or module? Do we accept eventual consistency here, and what does the reconciliation report look like when we do? Is this coupling tolerable given that the team on the other side is being reorganized next quarter? Models produce competent-sounding answers to all of these, because competent-sounding architectural prose is abundant in the training data. The answers are not grounded in your constraints, your people, your regulator, or your cost of being wrong five years out.

Accountability

When a release takes down payments, the incident review needs a chain of decisions with names attached to it. The agent wrote it is not a cause; it is a description of a gap in the process. Auditors, regulators and customers have not adjusted their expectations here and are not going to. Somebody signs. We keep that literal: a senior name signs every release, and that person carries the pager for it.

The estate nobody wrote down

This is the hardest one and the least discussed. In any system older than a few years, the load-bearing knowledge is not in the code. It is that the nightly job must finish before 04:15 because a partner SFTP drop expects the file. That the retry count is three because a vendor rate-limits at four. That an odd-looking branch exists because of a regulatory finding in 2017 and removing it is a compliance event. None of this is in the repository. A model reading the code infers a clean intent that was never there, and helpfully simplifies away the weirdness that was holding the system up. That is not a context-window problem to be solved by a larger window. The information does not exist in text.

Cost that moved rather than disappeared

The most common way an AI program disappoints is not that the generated code is bad. It is that the saving appears in one place and the new cost appears somewhere the accounting does not look. Three places, reliably.

Review load is the obvious one. If a change that took a day to write now takes an hour, but still takes ninety minutes to review properly, review is your throughput. Worse, it is a queue. Triple the arrival rate of diffs against a fixed number of senior reviewers and latency does not rise linearly — it rises sharply, and quality falls as reviewers begin to skim. In the first months of an agent-heavy workflow we typically see review load rise 20-30% before any net gain shows up. Teams that do not plan for that trough conclude the tools do not work at exactly the moment they were about to.

Reading unfamiliar code you did not write is slower than writing familiar code you did. That asymmetry is the entire cost structure of agent-assisted engineering.

Specification quality is the cost nobody budgets for. Vague requirements used to fail slowly and safely: an engineer with domain context noticed the gap, asked a question, and quietly filled it in. An agent does not notice. It fills the gap with the most statistically ordinary interpretation and produces two hundred confident lines on top of it. Ambiguity that a human used to absorb now becomes code, at volume. Written specification has gone from a good habit to a load-bearing input, and most teams are not practiced at it because they never had to be.

Evaluation is the third, and it is a genuinely new line item on the budget. Once agents are in the pipeline, every model upgrade, prompt revision and tool change is a change to your build system with no test suite attached. Without a harness of golden tasks drawn from your own repository, scored on outcomes rather than impressions, the claim that the new model is better is unfalsifiable. Building and maintaining that harness costs real engineering time. It is also the only thing that turns tool selection from taste into measurement.

The failure modes teams actually hit

These are not exotic. They are the ones we see repeatedly, on competent teams, with good tooling. What they have in common is that none of them announce themselves at merge time.

• Plausible-but-wrong. The pagination that drops the final page when the total is an exact multiple of the page size. The retry loop with no idempotency key, correct in every test and duplicating charges under a partial network failure. The timezone handling that is right in Bucharest and wrong for the customer in Auckland. These survive review precisely because they look like the code you expected to see.
• Context boundaries. Agents are strong inside a module and weak at the seams: cross-service contracts, feature flag interactions, online data migrations, the deploy pipeline. Retrieval over a large repository surfaces code that is lexically similar to the task, which is not the same set as the code that will break.
• Silent regressions, usually in cost rather than correctness. An ORM refactor that turns one query into N+1. A predicate wrapped in a function so the index stops being used. A serializer swap that changes float formatting for exactly one downstream consumer. Right answer, wrong price, discovered a week later on the latency graph or the invoice.
• Test theater. High coverage, low assertion value, generated in bulk, mocking the precise boundary where the bug lives. It raises a number your dashboard tracks and lowers the chance of catching anything.
• Silent duplication. Finding the existing helper requires whole-repository knowledge; writing a new one requires none. So you get the third date formatter, the second retry wrapper, and drift between them that surfaces months later as an inconsistency nobody can explain.
• Dependencies arriving quietly. Models propose package names with confidence, including names that do not exist — and attackers have started registering the popular hallucinations. Any agent-authored change that adds a dependency deserves a human decision rather than a rubber stamp.

There is a structural reason these get through. Human code review evolved to catch human mistakes, and human mistakes have a signature: the rushed section, the copy-paste with one variable left unchanged, the stretch where the style degrades because it was written at the end of a long day. Machine-authored code has none of those tells. It is uniformly confident and uniformly tidy across the whole diff, including the twelve lines that are wrong. Reviewers calibrated on human signal read that tidiness as care, and relax exactly where they should be paying more attention.

None of those is the expensive failure, though. The expensive failure is generated code that nobody understands, merged because it worked. The bill arrives eight months later, at 2am, when the engineer holding the pager has to reconstruct intent from a module that was never in any human head. Our rule is blunt: if no engineer on the team can explain a module without reading all of it, the team does not own that module, it hosts it. Hosted code accumulates quietly and then takes the system down.

What changes in daily practice

The teams getting durable returns changed how they work, not only what they type into. Six changes account for most of the difference.

The reason is simple enough to state. An agent is the most demanding colleague you will ever onboard: it reads the entire repository in seconds, retains nothing overnight, and never walks across the room to ask the person who wrote the thing. Every piece of context a human acquires by osmosis has to be present in the code, the tests or the written specification, or it does not exist. Teams that already had that discipline saw returns within weeks. Teams that did not spent two months building it first, and were right to.

1. 01The specification became an artifact. Before an agent starts, there is a written statement of the change, the interfaces it may touch, and what done looks like in a form a machine can check. Fifteen minutes of writing routinely saves two hours of reviewing the wrong thing.
2. 02Diffs got smaller on purpose. Cap the size of a change rather than the number of changes. A 200-line agent diff gets a real review. A 2,000-line one gets a scroll and an approval, and everyone involved knows it.
3. 03The repository became the interface. Fast deterministic tests, one-command environments, typed boundaries, error messages that say what to do next. These were always virtues. Now they are the difference between an agent that converges and one that flails, because they are the only feedback it gets.
4. 04Blast-radius rules replaced trust. Agents run freely on tests, documentation, migrations and isolated modules. Schema changes, authentication, money movement and anything touching a partner contract stay human-authored and human-reviewed, however good the generated version looks.
5. 05Review moved earlier. Reviewing the approach costs minutes; reviewing the implementation costs hours. More time now goes into the plan and less into the code, which inverts a decade of habit and takes a while to feel normal.
6. 06Evaluation became routine. Golden tasks from the real repository, run whenever a model, prompt or tool changes, scored on whether the output passes review and survives a week in production.

One further change is cultural and harder to engineer. The most valuable person on an agent-assisted team is no longer the fastest typist or the one who knows the framework best. It is the engineer who can decompose a fuzzy business requirement into verifiable units, and who reads unfamiliar code critically at speed. Those were always senior skills. They are now the skills — and the traditional path that produced them, juniors grinding through exactly the tedious work that agents now absorb, has a hole in it that our industry has not yet solved. We compensate deliberately: juniors review agent output alongside a senior, which turns out to be a faster way to learn what wrong code looks like than writing it yourself.

Measure the right station

Most reporting on AI productivity measures the part that got cheap. Lines generated, suggestion acceptance rate, time to first draft — all of these improve trivially and predict nothing about delivery. They are the equivalent of measuring how fast the first station on the line can run while the queue builds up behind it.

Measure downstream instead. Cycle time from specification to production, not to pull request. Change failure rate. Hours spent in review per merged change. Rework, meaning how often a merged change is materially rewritten within thirty days. Escaped defects by severity. If generation got faster and none of these moved, the work moved into a queue rather than out of the building, and it is considerably cheaper to learn that in month two than in year two.

Expect the curve to dip first. The opening six to ten weeks usually look worse on review load and no better on delivery, because the team is learning to specify, the repository is being made legible, and the evaluation harness does not exist yet. Judge the program at four months on shipped outcomes, not at four weeks on enthusiasm.

What this adds up to

None of this is an argument for restraint. The gains are real and they are not small — they are simply located somewhere other than where the demonstrations point. Sort the work by verifiability. Hand the machine everything bounded and checkable, at volume, without sentiment. Then spend the recovered senior hours on the parts that were always the hard parts: deciding what to build, understanding the estate you inherited, and standing behind what you shipped.

That is how we run delivery: a small senior core who hold the architecture and carry the pager, a wider network of engineers around them, and agents doing the bounded, verifiable work at a volume no team could staff for. The craft stays human; the throughput is machine. After twenty years and more than a hundred projects, the thing that determines whether a system is still working in five years has not changed at all — somebody has to understand it, and that somebody has a name.