When the cost of creating work approaches zero, what happens to the cost of understanding it?
Artificial intelligence has fundamentally changed the economics of producing work. Whether writing software, producing reports, drafting strategies or developing proposals, AI can generate in minutes what might previously have taken days.
But this creates a problem I call AI-Driven Confusion (AIDC). It emerges from the combination of two characteristics of AI-generated work: it is frequently nearly right, and it can be produced in extraordinary volumes.
The problem with nearly right
AI-generated work is often convincing. The writing is articulate, the structure is logical, and the code looks plausible. Most of it may even be correct.
The difficulty lies in the small proportion that isn’t.
A report that is 95% correct can contain a single flawed assumption that undermines its conclusions. A software application can pass a demonstration while concealing fundamental weaknesses in its implementation.
These errors are particularly troublesome because they are embedded within otherwise credible work. Finding them requires understanding the material, questioning its assumptions and exercising judgement.
In some circumstances, verifying AI-generated work can demand as much effort as producing it ourselves.
The problem with volume
The second problem makes the first considerably worse.
AI removes many of the constraints that previously limited how much work we could produce. A developer can generate thousands of lines of code in an afternoon. A consultant can produce a hundred-page strategy document in hours.
Yet the human capacity to review, comprehend and challenge that work has barely changed.
We have dramatically accelerated production without necessarily accelerating verification.
This creates an imbalance. Work accumulates faster than it can be properly understood. People begin to skim, trust appearances and accept conclusions without examining the reasoning behind them.
The result is a peculiar form of productivity: more output, but potentially less collective understanding.
Prose and code: an important distinction
Although AIDC affects both prose and software, there is a fundamental difference between them.
Prose is primarily interpreted by humans. We assess whether an argument is coherent, whether evidence supports its claims and whether its conclusions are sensible. There are tools that can assist, but much of the verification remains subjective.
Software is different.
Code is an executable artefact. Its behaviour can be observed, measured and tested against explicit expectations.
This creates an opportunity to address AIDC through engineering discipline.
Requirements can be expressed as executable acceptance tests. Automated testing can verify behaviour across thousands of scenarios. Static analysis, type systems, security scanning and continuous integration can provide further assurance.
Perhaps most importantly, tests can be developed independently of the AI generating the implementation. Otherwise, there is a danger that the same misunderstanding is reproduced in both the code and its tests.
None of this guarantees correctness. Tests can be incomplete, specifications can be wrong and unexpected behaviour can escape detection.
Nevertheless, software offers something prose generally cannot: the ability to establish substantial confidence in an artefact without examining every part of how it was produced.
From reviewing work to verifying outcomes
This suggests that the real opportunity for AI-driven development may lie beyond simply generating code faster.
We may need to fundamentally reconsider what humans should review.
Historically, software engineering has placed considerable emphasis on code readability, maintainability, peer review and understanding implementation details. These practices evolved when humans wrote the code and other humans were expected to maintain it.
But what happens when AI generates the implementation, AI can modify it, and independent automated verification can establish whether the resulting system behaves as intended?
Perhaps our attention should move towards specifying the right outcomes, establishing robust verification mechanisms and understanding the architecture and behaviour of the system, rather than scrutinising every line of its implementation.
This brings us to a question that challenges some deeply held assumptions about software engineering.
If AI-generated code delivers a system that behaves as expected, can be modified reliably as requirements change, and can be independently verified through comprehensive automated tests, do we ever actually need to read the code?
