Sonnet 5.5 is here. Is it worth switching?

The release of Claude Sonnet 5.5 on 28 September prompted me to compare selected recent models from Anthropic and OpenAI with their predecessors. I wanted to see how quickly they were improving, by how much, and whether the gains were useful beyond a launch announcement.

There is something reassuring about a goold old AI benchmark table. The names are technical, the numbers have decimal places, and the newest model usually appears in bold. After months of arguments about whether artificial intelligence will transform the economy or merely generate more emails, here is something that looks like an answer. Unfortunately, understanding the answer requires rather more work than admiring the table.

The results show substantial improvement in several useful areas. What’s harder is to turn those gains into a sensible conclusion about what a business should do next. Better scores are encouraging; deciding what they justify changing takes a little more thought.

Start with software development, where the evidence is unusually concrete. On SWE-bench Pro, which tests demanding changes to real software repositories, Claude Opus moved from 69.2% in version 4.8 to 79.2% in Opus 5 and 89.9% in Opus 5.5. That is a gain of 20.7 percentage points across three releases. You do not have to accept anyone’s predictions about artificial general intelligence to find that significant.

The less expensive Sonnet line is arguably more interesting. Sonnet 4.6 scored 58.1% on the same benchmark, Sonnet 5 reached 63.2%, and Sonnet 5.5 reached 81.3%. Most of that improvement arrived in the latest release. On this particular test, the newer Sonnet has a higher reported score than the previous Opus. Capability that recently belonged to the more expensive tier is becoming available further down the range. That matters if you intend to use these systems throughout a working day rather than occasionally ask one to impress you. [1]

Benchmark	Model sequence	First release	Second release	Third release	Overall gain
SWE-bench Pro	Claude Opus 4.8 to 5 to 5.5	69.2%	79.2%	89.9%	+20.7 points
SWE-bench Pro	Claude Sonnet 4.6 to 5 to 5.5	58.1%	63.2%	81.3%	+23.2 points
SWE-bench Pro	OpenAI GPT-5.5 to GPT-5.6 Sol to GPT-6 Astra	59.4%	64.6%	Not established	Not calculable
DeepSWE v1.1	Claude Opus 4.8 to 5 to 5.5	59.0%	68.8%	74.2%	+15.2 points
DeepSWE v1.1	OpenAI GPT-5.5 to GPT-5.6 Sol to GPT-6 Astra	67.0%	72.7%	74.1%	+7.1 points

Scores should be compared within the same benchmark. Claude figures use Anthropic’s system cards; OpenAI figures use its release tables. Different evaluation setups mean these are reported trends, not a controlled head-to-head experiment. OpenAI’s GPT-5.6 comparison reports 59.4% for GPT-5.5 on SWE-bench Pro, whereas its original launch reported 58.6%; the table uses the later paired comparison. A comparable Astra score on SWE-bench Pro was not established. [1][2]

OpenAI’s progression tells a slightly different story. On DeepSWE, a test of longer software-engineering tasks, GPT-5.5 scored 67.0%, GPT-5.6 Sol reached 72.7%, and GPT-6 Astra reached 74.1%. On BrowseComp, which tests difficult web research, the sequence was 84.4%, 90.4% and 91.5%. Those are useful gains, although the last step looks modest. The DeepSWE rows put both companies on the same named test, although they still use their respective evaluation setups. Opus 5.5 and Astra finish almost level on the reported scores; a tenth of a percentage point is no basis for declaring a winner. [2]

Then look at a newer evaluation. On Terminal-Bench-Science, which tests scientific workflows involving code and analysis, Astra’s launch comparison reported 64.6%, against 22.4% for GPT-5.6 Sol. Claude Opus improved from 29.0% to 58.7% in its latest comparison. These are large advances in the reported ability to carry out a sequence of research tasks. A model can make a modest gain on one established test and a much larger one on a different kind of work. There is no single improvement percentage waiting to be extracted from all this, however convenient one would be for a presentation. [2][3]

Terminal-Bench-Science 0.1	Previous model	Newer model, cited comparison	Gain
OpenAI: GPT-5.6 Sol to GPT-6 Astra	22.4%	64.6%	+42.2 points
Claude: Opus 5 to Opus 5.5	29.0%	58.7%	+29.7 points
Claude: Sonnet 5 to Sonnet 5.5	Not established	59.9%	Not calculable

Sonnet 5.5’s scientific-workflow score comes from its system card. Anthropic reports a standard error of about 4.7 points for Sonnet and 4.8 for Opus 5.5, so their 1.2-point difference is not convincing evidence of superiority. [1]

For readers whose day involves documents and analysis rather than software repositories, professional-work evaluations provide another useful comparison. GDPval-AA assesses professional deliverables; AA-Briefcase tests longer professional workflows. Both use rating points, not percentages, and the versions matter.

Model	GDPval-AA v2.1	AA-Briefcase v1.1
Claude Opus 5.5	1846	1822
Claude Sonnet 5.5	1844	1811
OpenAI GPT-6 Astra	1542	1569
OpenAI GPT-6 Sol	1487	1483

These published comparisons place the two Claude models close together and above the OpenAI models on these particular tests. They do not tell us that one model makes an employee a corresponding percentage more productive. The results also illustrate why a universal winner is elusive: Astra has the higher reported scientific-workflow score, while Claude leads this professional-work comparison. These are the models covered by this review, rather than a live leaderboard. Anthropic also flags a fixed structured-output bug in the Sonnet prerelease evaluation and a fixed image-understanding bug in GPT-6 Sol that might not yet be reflected in these ratings. The evaluators expect small or no major effects, but these are still scores for the tested deployments. [1][3][4]

The footnotes become essential when you try to extend the comparison. Benchmark versions change. Models are given different tools, different time limits and different amounts of reasoning. An older model can acquire a new score because someone has tested it differently. Anthropic’s published Sonnet 5 result on Humanity’s Last Exam, with tools, was 57.4% in its June system card and 54.9% in the September comparison. That does not establish that the model spent the summer becoming less intelligent. It establishes that the two numbers need explaining before they enter the same chart. [1]

There is a good reason for some of this movement. Tests need to evolve as systems become more capable, and better evaluation methods should replace worse ones. But the result is an awkward bargain: the industry needs a moving test to measure its progress, while the reader needs a fixed test to understand that progress. A polished graph can conceal the tension remarkably well.

Even the instruction to give a model more time to think turns out to have qualifications. Sonnet 5.5 scores 52.1% on FrontierCode at its extra-high reasoning setting, but 46.2% at maximum effort. Anthropic explains that the more intensive setting sometimes produced timeouts or changes beyond the requested scope. The additional work could be competent and still make the answer less acceptable. Anyone who has commissioned a small software change and received a proposal to rebuild the platform will recognise the distinction. [4]

This is where the comparison starts to become useful beyond the leaderboard. A business is buying an outcome with boundaries: fix this fault, produce this analysis, deliver this feature. It has limited use for a system that completes a more ambitious task than the one it was given, then requires somebody to work out which parts to undo. Scope discipline belongs in the definition of capability.

Nor do all scores improve. Opus 5.5 recorded 77.8%, below Opus 5’s 80.6%, in Anthropic’s internal Toolathlon evaluation, which tests workflows involving multiple tools. Safeguard-related stops count as failures in that setup. Whether a particular stop represents an inconvenience or a valuable restraint depends on the task. The percentage alone cannot settle it, or establish a general decline in capability. [1]

For a management team, the next question is therefore more demanding than which model has the highest score. Suppose AI helps engineers finish a feature sooner. Does review also finish sooner? Can the organisation release it, can customers use it, and will anyone pay for it? If the work spends the saved week waiting for approval, the coding improvement is real but the commercial benefit remains hypothetical.

The useful measure is the cost of reaching an accepted result, including the human effort required to check and correct it. For a software business, that means following work through to release and customer use. In a strategy built around selling additional functionality, the eventual evidence is earlier paid adoption and additional revenue. Counting generated code somewhere near the beginning of that chain will not tell you whether the rest of it moved.

That leaves a more immediate question for anyone following these releases: are the differences between the leading models still significant enough to justify choosing one over another? If each can produce a useful first draft, analyse a document and write competent code, how much attention should we give to which one has moved a few places up the leaderboard?

The results do not establish that the models are interchangeable. They do suggest that the answer depends on where you place the finishing line. If two models both produce work you can accept with a quick check, a higher benchmark score may add little to your day. Speed, cost, ease of use and access to the tools you already work with could matter more. Familiarity has value too: switching means learning a new system’s habits and finding out where it needs supervision.

For harder tasks, a difference can still be decisive. One model might complete work that another repeatedly fails, or require substantially less correction. Even a small reliability advantage could matter when the same task runs thousands of times. But that advantage needs to appear in the work itself. A narrow lead on an unrelated benchmark is a thin reason to disrupt a process that is already working.

There is a practical way to make that decision without waiting for the industry to agree on a universal scoreboard. Keep a fixed set of representative tasks, including work the current model struggles with. Record the model and settings, then compare acceptance, elapsed time, correction effort and failures. Public benchmarks help decide which alternatives deserve a trial; your own results determine whether a switch is worthwhile. If you cannot find a meaningful improvement, staying put is a reasonable decision.

Both companies are producing systems that can handle more demanding work, and Sonnet’s progress suggests that useful capability is spreading beyond the premium tier. As generating a plausible result gets easier, knowing whether it is the right result becomes more valuable. The business advantage may increasingly belong to organisations that can specify, verify and use the work quickly enough to benefit from it. That is also a reason to spend less time following every change at the top of the leaderboard: you can afford to wait until an improvement addresses a limitation you actually have.

The Eight Hours We Haven’t Planned For

In 1930 John Maynard Keynes made one of the most famous predictions in economics. Technological progress, he thought, would make his grandchildren so productive that they might need to work only 15 hours a week.

The prediction is usually remembered because it was wrong. People became vastly richer, but they continued to work.

Less often remembered is that Keynes was not entirely relaxed about the alternative. If technology eventually solved what he called the economic problem, humanity would encounter another one: what to do with its freedom. People had been trained for generations to strive, he argued, and might prove surprisingly bad at doing anything else. Those with money but few duties or ties did not strike him as particularly encouraging examples of life after work.

Two years later, in an Austrian village called Marienthal, researchers stumbled across a darker version of the same problem.

The village’s textile mill had closed, leaving most families dependent on unemployment support. Poverty obviously dominated what followed, which makes Marienthal a poor model for an affluent automated future. But something else happened that fascinated the researchers. Time itself seemed to change.

The unemployed men walked more slowly. Many stopped repeatedly while crossing the village. Most no longer wore watches. Library borrowing fell rather than rose. Presented with an abundance of time, people did not automatically fill it with reading, hobbies or civic life. The structure that had made time useful had disappeared.

Nearly a century later, artificial intelligence raises a much larger version of Keynes’s question.

Most discussion about AI and employment is understandably economic. Which jobs disappear? How quickly does productivity rise? Who owns the machines? How should the income they generate be distributed?

But suppose, eventually, that those problems can be solved. Imagine a society rich enough to provide people with comfortable lives even when relatively little human labour is required.

What will everyone actually do all day?

For most adults, work occupies roughly eight hours of every weekday. For children and teenagers, school does much the same. Add commuting, preparation, homework and the mental space occupied by both, and a remarkable proportion of human existence is organised around the requirement to be economically useful.

Now imagine that requirement begins to disappear.

Not because useful things no longer need doing, but because machines can do more of them, more cheaply and eventually better. They diagnose illnesses, write software, analyse businesses, prepare contracts, design buildings and teach mathematics. If robotics makes comparable progress, much physical labour may follow.

Intelligence becomes a utility. Eventually labour may begin to resemble one too.

The economic consequences would be enormous. The psychological consequences may be stranger.

Work does much more than produce income. It tells people when to wake up. It supplies colleagues, deadlines, routines and mild competition. It creates achievements and disappointments. It provides status and identity. It gives someone a reason to leave the house and, perhaps most importantly, gives other people reasons to expect them to arrive somewhere.

School performs a similar function for the young. It is supposedly an institution for transferring knowledge, but it is also a machine for structuring childhood. It puts several hundred young people in the same place at the same time. It forces them to coexist, encounter people they did not choose, play sport, sit examinations, perform plays, make friends, lose arguments, cope with boredom and occasionally discover that they are good at something they had never considered trying.

Remove much of work and school and society does not merely lose two institutions.

It loses much of its timetable.

Retirement gives us the closest existing approximation. Someone reaches sixty or sixty-five and, often quite abruptly, discovers that the institution which has organised much of adult life has disappeared.

Some flourish. They have family, friendships, travel, voluntary work, sport and projects waiting for them. Others discover that work had quietly been supplying much of their social world and sense of identity.

But a person retiring at sixty five has had four decades to work out who they are. They have accumulated relationships, interests and obligations. With luck, they know what gives shape to a Tuesday when nobody is paying them to be somewhere at nine o’clock.

An eighteen year old has not.

That may be the harder problem.

If an eighteen year old enters a world in which employment is optional, scarce or simply unnecessary, what exactly are they supposed to do with the next seventy years?

At that age most people have barely begun to discover what they care about. School, university and work provide tracks to follow while identity gradually forms. They impose sequence. Study this. Qualify. Find work. Become better at something. Change direction. Perhaps raise a family. Eventually retire.

Remove the tracks and society may give young people unprecedented freedom before they have developed the judgement required to use it.

The immediate alternatives are not wholly reassuring.

An enormous industry already exists whose purpose is to capture unclaimed attention. Social networks, games, streaming services and personalised entertainment compete continuously for spare minutes. If work releases another eight hours a day, some of the richest companies in history will have every incentive to occupy them.

The technologies will also become better at the job. AI will be able to create endless games, stories, relationships and virtual worlds tuned precisely to the psychology of one individual. Boredom may become technologically obsolete.

A society could therefore achieve something close to material abundance while millions of people spend much of their lives inside experiences optimised to stop them leaving.

Yet boredom may not be the deepest difficulty.

Decades of research into unemployment suggest that employment supplies several things besides pay: structure to time, social contact, status, regular activity and participation in a collective purpose. Marie Jahoda, one of the researchers at Marienthal, later called these the latent functions of work. Modern research still finds links between these functions and mental wellbeing.

But there may be something underneath all of them.

People want to matter.

Psychologists use that word in a fairly precise way. Mattering means not simply believing that your life has purpose, but feeling significant to other people. One component is dependence: the knowledge that somebody relies on you. Another is the feeling that your actions genuinely make a difference.

This suggests that the usual discussion about the end of work may be asking the wrong question.

We ask what people will do.

Perhaps we should ask who will need them.

There is an important difference.

I can decide that learning Portuguese is my purpose. I can spend five years becoming an excellent pilot. I can go to the gym every other day, ride horses or study astronomy. All could make my life richer.

But if I decide not to do any of those things tomorrow, nobody necessarily suffers.

Work is different because it often makes our actions consequential to other people. A patient expects the doctor. A pupil expects the teacher. A customer expects the plumber. Your colleague needs the report because she cannot finish something until you do.

Even an ordinary job embeds a person in a small network of dependence.

The uncomfortable possibility is that AI does not merely make us less necessary to the economy. It makes us less necessary to one another.

If an AI can teach the child, diagnose the patient, repair the machine, give the legal advice, drive the passenger and perhaps eventually provide companionship, many of the practical reasons humans have historically had to depend upon other humans begin to weaken.

Materially, this could be wonderful.

Socially, it presents a peculiar problem.

A world in which nobody needs to do very much for anyone else may be comfortable while also making it harder to answer one of the oldest human questions: where would there be a hole if I disappeared?

AI could even simulate the answer. An artificial companion could tell you that it missed you. A game could give you a vital mission. A virtual organisation could provide status, promotion, difficulty and praise.

But there is a difference between being told that you matter and knowing that something outside yourself actually depended upon your turning up.

That distinction may become increasingly valuable.

The same is true of time.

Work and school do something so familiar that we barely notice it: they synchronise millions of lives.

Monday morning means roughly the same thing to everybody. Evenings are when many people stop working. Children have school holidays together. Saturday afternoon works as a social institution because large numbers of people are available at the same time.

Research into time use shows that this synchronisation matters. Couples actively coordinate working hours to create shared leisure, and people whose routines are badly out of step with society often pay a social cost.

This leads to a curious possibility.

A post work society could have an abundance of individual free time and a shortage of common time.

If everybody is free whenever they like, everybody may choose differently. The rhythms that currently coordinate us could weaken along with the jobs that created them.

Saturday means something partly because Monday exists.

This is why simply building more leisure facilities does not quite solve the problem.

A future rich in sports centres, parks, workshops, libraries and virtual worlds might offer people an extraordinary range of activities. But activities are not the same as obligations, and private leisure is not the same as shared life.

The institutions that matter most may therefore be those that create two things work currently supplies almost accidentally: mutual dependence and common time.

A football team does this. Training is on Wednesday because everyone has agreed that Wednesday is when they train. If the goalkeeper does not appear on Saturday, ten other people notice.

An orchestra does it. So does a theatre company, a voluntary organisation, a scientific project, a community garden, a care network or a group restoring a woodland. Their value is not simply that they keep people busy. They create roles in which other people rely on you.

Perhaps that is the infrastructure a post-work society would actually need.

Not organised leisure, exactly. Something closer to organised significance.

It would need institutions capable of creating what might be called voluntary necessity: commitments freely chosen, but real enough that once chosen they impose duties.

There is a paradox here. We imagine freedom as the removal of obligations. A society liberated from compulsory work might discover that a satisfying life requires putting some obligations back.

The difference is that people could choose which ones.

Instead of being compelled by economic necessity to spend forty hours a week doing whatever the labour market happens to reward, people might have much greater freedom to choose the communities, skills, causes and people to whom they wish to become useful.

That could be one of AI’s more attractive consequences.

It might allow societies to distinguish for the first time at scale between what is economically necessary and what is humanly necessary.

Looking after an elderly neighbour may generate little GDP. Raising children, maintaining a community garden, mentoring a teenager or helping run a local charity can be economically marginal while being deeply consequential to the people involved.

Our present economy often gets this backwards. Some of the activities people consider socially valuable are poorly paid, while some highly paid work leaves those doing it wondering whether anything would change if the job disappeared.

AI could deepen that absurdity, or help us escape it.

Education would then have to change too.

If an infinitely patient AI tutor can teach algebra, chemistry or French, schools may need to spend less time transmitting information and more time teaching young people how to participate in human institutions.

How do you become someone others can rely upon? How do you cooperate, lead, lose, persist, make friends, resolve conflict and keep a commitment after the enthusiasm that produced it has disappeared?

These are not secondary skills in a post-work society. They may become the central ones.

A good school might still teach history and mathematics, but its deeper task would be to help a young person discover not merely what interests them but where they can become useful to others.

That is harder than teaching algebra. Algebra has a syllabus.

A life does not.

There is good reason nevertheless to be optimistic.

Human beings have never shown much enthusiasm for doing nothing. When released from necessity they repeatedly invent difficulty. They run marathons, climb mountains, become obsessive gardeners, learn musical instruments, compete at sport and spend years mastering skills with almost no economic value.

The problem is not that humans lack the desire to strive.

It is that the market has spent several centuries performing the useful function of connecting that striving to a social structure.

The next task may be to make those connections without requiring a wage.

That is why retirement is an incomplete model for the future. A sixty five year old often enters retirement already embedded in a lifetime of relationships and obligations. There may be children, grandchildren, old colleagues, clubs, neighbours and interests accumulated over decades. Work disappears, but the person does not start again from nothing.

An eighteen year old may.

So perhaps the most important question raised by a post-work society is not how an eighteen-year-old fills seventy years of leisure.

It is how, over those seventy years, they become entangled in the lives of other people.

For most of human history necessity did much of that work for us. Families depended upon one another because survival demanded it. Communities depended upon cooperation. Employers needed workers. Workers needed wages. Scarcity forced humans into relationships of mutual reliance whether they liked it or not.

AI may loosen some of those bonds by solving the scarcity that created them.

That should still be welcomed. There is nothing intrinsically noble about drudgery, poverty or having no choice about how to earn a living.

But removing economic necessity does not remove the human need to be consequential.

If anything, it makes that need more visible.

The deepest challenge of the AI age may therefore turn out not to be finding enough things for humans to do. Machines and entertainment systems can provide endless things to do.

The harder task will be constructing a society in which people still have reasons to need one another.

For most of modern history we have asked young people: what can you learn to do that somebody will pay you for?

An eighteen year old in an age of abundant intelligence may face a much larger question.

Who will you choose to become useful to?

A good society would make sure there are many answers.

What happens when intelligence becomes a utility?

A friend and I were talking about AI on the bus to last weekend’s football game and, as usual, ended up on the familiar questions. How good are these systems getting? Which jobs will go first? What happens if they eventually become better than us at most things?

As the bus wound its way through the inevitable traffic outside the stadium, we got onto a different question. If the cost of producing useful intelligence keeps falling, what happens when it becomes so cheap and readily available that we begin to treat it like a utility?

For most of human history, useful intelligence has been expensive. Diagnosing an illness, interpreting a contract, writing software or analysing a company usually requires somebody who has spent years learning how to do it. Societies responded by building institutions around that scarcity. Universities produced expertise, professions certified it and companies assembled it.

When you pay a good lawyer or engineer, you are not really buying an hour. You are buying a small slice of everything that lawyer or engineer has learned to make that hour useful.

The cost of intelligence is already falling quickly. When GPT-4 appeared in 2023, OpenAI charged $30 per million input tokens and $60 per million output tokens. More capable models now cost a fraction of that. The comparison is not perfectly like for like, but the direction is clear enough. More intelligence is being delivered for much less money.

Electricity offers a useful precedent. Before electrification, factories had to produce power close to where it was consumed. Electricity changed that. Once power became reliable and cheap, firms stopped designing around its scarcity and started assuming it would simply be there.

The important effect was not cheaper machinery. It was what followed. Elevators made tall buildings practical. Refrigeration transformed food distribution. Washing machines changed domestic life. Computing eventually grew on top of the same infrastructure. Once power was abundant, whole categories of activity that had previously made little sense became viable.

AI may do something similar to cognition.

There is a great deal of thinking that simply does not happen today because it costs more than it is worth. Companies do not investigate every small opportunity. Doctors cannot ask ten specialists to review every patient. Scientists cannot explore every plausible hypothesis. These limits are so familiar that we barely notice them.

However, lower the cost far enough and some of those limits start to disappear. Imagine every medical diagnosis being looked at from several directions. One system checks drug interactions. Another looks for unusual conditions. Another scans the latest research. A fourth is told to challenge the conclusions of the others.

That does not necessarily make the doctor redundant. It makes something that is currently exceptional, several expert opinions on one case, routine.

The same logic extends much further. A small business might have something resembling a lawyer, finance team, developer and market analyst available whenever it needs them. A student could keep asking for a different explanation until a difficult idea finally clicks. A scientist might explore a hundred possibilities before deciding which are worth testing in the real world.

This is why it is important not to get trapped solely in the immediate questions about AI: which jobs disappear, which tasks are automated, which professions are disrupted. We also need to think about what a post-AI economy looks like once intelligence is cheap and widely available.

History suggests that when one thing becomes plentiful, something else becomes scarce.

The internet gave us almost unlimited information and left us fighting for attention. Digital cameras made photographs virtually free without making everyone a good photographer.

If AI gives us almost unlimited analysis, then another analysis may not be worth very much.

A management team may soon be able to generate a hundred plausible strategies before breakfast. The difficult part will still be choosing one and living with the consequences.

The same is true of financial analysis. If models and reports can be produced almost endlessly, value shifts towards people who can explain what the numbers mean, spot dubious assumptions, understand the business behind them and help somebody decide what to do next. Good analysts may spend less time calculating and more time interpreting.

Producing an answer is also not the same as knowing that it is right. AI might produce a legal opinion in seconds, while somebody still has to decide whether they trust it enough to act on it. Lines of software can be generated quickly, but proving that the result is safe enough to run a bank is a very different problem.

So I do not think expertise simply disappears. It changes shape. Knowledge and analytical skill matter, but so do experience, credibility, judgement and responsibility. AI can make the first two much cheaper. The others are more stubborn.

This could alter the shape of companies too. Organisations exist partly because useful knowledge is scattered across people and coordinating those people is costly. If AI reduces the cost of both expertise and coordination, some activities that currently require a large organisation may no longer do so.

A small company may gain access to capabilities that once required entire departments. Large organisations will still enjoy advantages in capital, reputation, distribution, proprietary data and physical assets. But the ability to employ large numbers of clever people may no longer confer quite the same edge.

By the time my friend and I reached the football, fortunately, we had stopped talking about AI. Like it or not, football is already drenched in analysis: expected goals, tracking data, tactical models, performance statistics. AI will provide vastly more of it.

Yet it is hard to imagine that wealth of data making the match itself more meaningful for the fan. There is still only one game taking place. Nobody knows exactly what will happen. If your team misses a penalty, a computer can simulate a thousand versions in which it goes in, but none alters the afternoon you actually experienced.

Perhaps that hints at another consequence of abundance. Some things remain stubbornly scarce.

And this is where the electricity comparison breaks down.

Electricity is passive. It can power a television but does not tell you what to watch. Intelligence can. An AI helping with a decision can also choose which evidence to show, how to frame the question and which possibilities appear sensible.

That makes intelligence a peculiar sort of utility. We may come to depend on it while it is also helping to shape the way we understand the world.

Questions that sound technical today may therefore become important. Who owns these systems? What are they optimising for? How independent are two supposedly different opinions if both depend on the same underlying models? Who takes responsibility when everyone follows the answer because everyone assumes somebody else has checked it?

Many of our institutions were built for a world in which intelligence was difficult to find and expensive to organise. Schools, professions, companies and bureaucracies all reflect that scarcity in different ways.

AI will not sweep them away. But if competent intelligence becomes something we can summon almost without thinking about the cost, it would be surprising if they all continued to look much the same.

We spend an enormous amount of time debating how intelligent AI will become. I think we need to spend much more time thinking about the world that follows if intelligence becomes cheap and abundant.

Technology can change very quickly. Institutions, education, companies and social norms usually do not. If intelligence really is becoming a utility, the danger is not simply that we underestimate what AI can do. It is that the technology arrives long before we have worked out what we want to do with it.

The AI Agent Revolution: Two Years On

In November 2024 I wrote an article called The AI Agent Revolution. I thought I had a reasonable idea of what was coming.

The argument was that AI was entering what I called “Phase 3”. Phase 1 had been conversational AI: systems that could talk, write and answer questions. Phase 2 was reasoning. Phase 3 would be different because AI would stop merely telling us what to do and start doing things for us.

At the time, this was mostly prediction. Microsoft had demonstrated Magentic-One, an experimental system in which several AI agents worked together. Anthropic had given Claude the slightly unnerving ability to look at a computer screen, move a mouse and click on things. OpenAI was rumoured to be developing something called Operator.

Nearly two years later, the interesting thing is not that the prediction was right. Agents have arrived. It is that they are turning out to be rather different from the autonomous AI workforce I had imagined.

They are both more capable and more constrained.

Consider Anthropic’s original computer-use demonstration. In October 2024 Claude 3.5 Sonnet scored just 14.9% on OSWorld, a benchmark that asks an AI to operate ordinary software through a computer interface. Humans scored around 72%. Watching Claude use a computer was remarkable, but the score was a useful corrective. It was a glimpse of something interesting rather than something you would necessarily want anywhere near your finance system.

Claude Opus 5.5 now reports 81.8% on OSWorld 2.0. The benchmark and methodology have changed, so treating 14.9% to 81.8% as a straightforward improvement would be misleading. Even so, something fairly substantial has happened in between.

Software engineering tells a similar story.

Claude 3.5 Sonnet scored 49% on SWE-bench Verified in October 2024. The test asks an agent to take a real GitHub issue, understand an unfamiliar codebase, change it and produce a working fix. By August 2025 GPT-5 was scoring 74.9%. Frontier models became so good at the benchmark that OpenAI eventually said SWE-bench Verified was no longer particularly useful for measuring the best coding systems.

So the tests got harder.

On Terminal-Bench 4.0, which involves complex terminal-based work including software engineering, system configuration and data analysis, GPT-6 Astra scores 57.9%, compared with 37.3% for GPT-5.6 Sol and 55.8% for Claude Fable 5.1.

Research agents have improved just as dramatically. When OpenAI introduced BrowseComp in April 2025, it deliberately constructed questions whose answers were difficult to find on the web. GPT-4o with browsing managed 1.9%. OpenAI’s Deep Research system reached 51.5%. ChatGPT Agent subsequently reached 68.9%. GPT-6 Astra now reports 91.5%.

I am increasingly wary of apparently miraculous benchmark improvements, however. Once a benchmark becomes public, the relationship between the test and what it is supposedly testing gets complicated. Anthropic has even documented Claude recognising BrowseComp and finding leaked answers online. An AI cheating on its AI benchmark is perhaps progress of a sort, although not quite the sort the benchmark designers had in mind.

The exact numbers therefore deserve some caution. The underlying change is harder to explain away. These systems have become much better at searching, changing strategy, using tools and persisting when their first attempt fails.

The measurement I find most interesting comes from METR because it asks a different question. Instead of testing whether an AI can answer something difficult, it asks how long a task an agent can take on before it becomes unreliable.

METR estimates how long a skilled person would need to complete a coherent task and then measures whether an agent can complete it successfully. Historically, the length of task frontier agents can complete with 50% reliability has been doubling roughly every seven months.

By May 2026 an early Claude Mythos system had reached a 50% task horizon of at least 16 hours on METR’s predominantly software and technical tasks. METR actually ran into a slightly unusual measurement problem: beyond 16 hours there were too few sufficiently difficult tasks left to estimate the result reliably.

That does not mean an AI can now turn up on Monday morning and reliably do somebody’s job until 5pm. METR’s tasks are unusually well specified and mostly technical. Real work contains ambiguity, politics, missing information, badly written emails and meetings whose purpose nobody can quite remember.

Still, sixteen hours matters. It suggests that we are moving from AI completing isolated tasks to AI sustaining useful work across long sequences of decisions.

And this is where things get more complicated.

Zapier’s AutomationBench puts agents inside simulated businesses and asks them to complete workflows across CRMs, inboxes, calendars and the other systems used in sales, marketing, finance, HR, support and operations. GPT-6 Astra currently completes 41.4% of those workflows successfully. Claude Opus 5.5 manages around 40%. GPT-5.6 Sol manages 18.1%.

Forty-one per cent is simultaneously an extraordinary result and a terrible one.

A couple of years ago, an AI independently completing four out of ten unfamiliar business processes would have sounded like science fiction. If I were actually running those processes, though, I would be rather interested in what happened to the other six.

That, I think, explains much of what has happened with agents.

The technology has become powerful enough to do useful work, but not reliable enough to be given unlimited freedom. So businesses have started building systems around it.

An agent today is no longer simply a language model with a clever prompt attached. It can use browsers, terminals, files, databases and business applications. It can maintain context while it works, choose tools and sometimes recover from its own mistakes.

OpenAI’s Operator, still a rumour when I wrote the original article, appeared in January 2025 and could operate a browser itself. Since then, that sort of capability has been absorbed into broader systems combining browsing, research, coding, files and connections to other applications.

The model is increasingly only part of the product. The interesting questions are what the agent can see, what tools it can reach and, above all, what it is allowed to do.

Real deployments make this clearer.

Siemens now uses two AI agents to deal with incoming sales leads. One talks to potential customers and another qualifies them before suitable opportunities reach salespeople. About 2,500 leads a month across 132 countries pass through the system, which Siemens says went from prototype to production in six weeks.

But look more closely at how it works. The AI conducts the conversation while roughly 50 predefined business rules determine the qualification outcome. There are also boundaries around what the agent is allowed to discuss.

That is not an autonomous digital salesperson roaming freely through Siemens. It is something more carefully engineered, and probably more useful.

Uber has taken a similar approach with Uber for Business. Its agent follows up with leads, answers questions and revives conversations that might otherwise have gone cold. Conversion from those emails is reported to be around 60% higher than Uber’s previous organic baseline.

Customer service has gone further. Intercom says its Fin agent is used by more than 12,000 businesses and resolves about 76% of the conversations in which it is involved on average.

These are no longer demonstrations. McKinsey’s latest survey found that 40% of respondents from companies with more than $1 billion in annual revenue said they were scaling AI agents, up from 27% a year earlier.

What has not scaled at quite the same rate is trust.

A chatbot producing a poor answer is irritating. An agent sending the wrong information to thousands of customers, exposing confidential data or incorrectly updating a financial system is a different class of problem. The more useful agents become, the more consequential their mistakes become too.

This has created a market I paid much less attention to when I wrote the original article: the machinery around the agent.

Companies such as Dynamo AI are building control layers that watch what goes into models, what comes out and what an agent is allowed to do. DynamoGuard, for example, can detect sensitive-data leakage, hallucinations, prompt injection and policy violations, then block, sanitise or escalate activity in real time.

Credo AI approaches the problem more from the governance side, helping organisations keep track of agents, map them against regulatory and corporate requirements and monitor what they do once deployed.

I suspect this layer will become much more important.

Traditional technology governance assumes that a system is designed, tested, approved and then operated more or less as expected until somebody changes it. That assumption becomes uncomfortable when the software can make thousands of individual decisions every day and can decide for itself which tool to use next.

A company therefore needs to decide what data an agent can see, which systems it can access, what it can change, when it needs permission and when it should simply stop.

Those rules have to exist in the software. A policy document sitting in SharePoint will not save you.

This has also changed my view of the “autonomous workforce” I wrote about in 2024.

I imagined agents gradually becoming something like digital employees, with humans remaining in the loop to supervise them. There is some of that. But the more interesting pattern is that agents are beginning to rearrange jobs rather than simply replace them.

In sales, an agent can research a customer, start the conversation and qualify the opportunity. The salesperson spends more time on the relationship and the deal.

In customer support, an agent can absorb large volumes of routine questions. People deal with the unusual, sensitive or commercially important cases.

A coding agent can write, test and debug substantial amounts of software. Engineers can spend more time deciding what should be built, thinking about architecture, reviewing what the agent produced and dealing with the difficult exceptions.

The same thing is beginning to happen in analytical work. An agent can collect information, reconcile data, build an initial model, test assumptions and produce a first version of an analysis. The scarce part of the job then moves elsewhere: deciding whether the assumptions make sense, noticing what is missing and explaining what the numbers mean in the real world.

If AI makes the mechanical parts of analysis much cheaper, companies with deep subject expertise and strong client relationships may not need fewer people doing the same amount of work. They may be able to produce more with the same team, turn work around faster and take on assignments that previously would not have been economical.

Two years ago, I thought the important question was whether AI agents would become capable enough to do real work.

I don’t think that is the question anymore. They plainly can.

The more useful question is which pieces of work we are prepared to hand over to them. The best candidates seem to have a few things in common: the task has reasonably clear boundaries, success can be measured, mistakes can be detected and, ideally, a bad decision can be reversed before too much damage is done.

Then comes the harder part. How much authority should the agent have? What can it see? What can it change without asking? When does a person need to intervene?

That is rather less exciting than predicting an army of autonomous digital employees. It is also much closer to what businesses actually need to work out.

Phase 3 AI arrived. The surprise is that the most interesting question is no longer how intelligent the agents are becoming, but how much of our world we are prepared to let them touch.

We’re racing towards AGI. But what do we actually want AI to do?

The discussion about artificial intelligence felt rather different this week. Jack Clark, the Brighton-born co-founder of Anthropic, was talking about the possibility that we may eventually need to slow the development of the most powerful AI systems, and even about independently verifiable ways of shutting them down. Then, almost in counterpoint, Nature reported on a “virtual biotech” involving tens of thousands of AI agents working together on drug discovery and identifying a promising new approach to lung cancer treatment. One story was about restraint, the other about possibility. Put together, they seem to capture a choice we are beginning to face.

We spend a great deal of time talking about the race towards Artificial General Intelligence: which company will get there first, which country will dominate it, how much more capable the next model will be. There are perfectly understandable commercial and geopolitical reasons for this. But the race itself can start to dictate the objective. In trying to build an intelligence capable of doing almost everything, we may be paying too little attention to a simpler question: what do we actually want all this intelligence to do for us?

Cancer seems an obvious place to start. Rather than directing every new increment of computing power towards making one general purpose model slightly better at everything, imagine deliberately creating hundreds, perhaps eventually thousands, of specialised AI agents focused narrowly on understanding and treating it. Some could concentrate on individual cancers, others on genetics, protein interactions, immune response, drug discovery, pathology, imaging or clinical trials. They would work alongside scientists, doctors and laboratories, testing ideas against evidence, learning from failed experiments and sharing what they discover.

The important point is that this isn’t just theoretical. Large numbers of AI agents collaborating on scientific problems are already beginning to appear in real research; the Nature article is one early example. So extend the idea all the way to the frontier. Apply the same intensity to Alzheimer’s, antibiotic resistance, fusion energy, energy storage, food production, clean water and other stubborn problems humanity has been trying to solve for decades. The attraction is not simply that these are worthy causes. Narrowing the AI mission may also be a sensible way to use increasingly powerful AI while we learn how to make it safer.

That matters because we do not have to choose between racing ahead and stopping altogether. There is a middle course in which we continue to improve the intelligence, and continue putting it to work on difficult real world problems, while being more cautious about the autonomy we give increasingly general systems. Capability and permission do not have to advance at the same speed. A system can become dramatically better at science, medicine or engineering without automatically being given greater freedom to acquire resources, rewrite itself or act independently across the world. Narrowing the mission, at least for a time, could give us something that may soon become scarce: room to think.

There is a larger version of the idea too. What if the major powers agreed that a handful of these problems were simply too important to become another winner-takes-all technological race? The United States might lead a major international effort on cancer. China might commit enormous resources to fusion or energy storage. Europe could focus on neurodegenerative disease, with Britain, India, Japan and others taking responsibility for different challenges. The precise allocation is not the point. Each country would bring its best scientists, laboratories, computing power and AI systems to bear on a defined human problem, then share what it discovers.

Of course, this would be difficult. Countries compete, companies have shareholders, governments have security concerns and intellectual property has enormous value. None of that disappears because we might prefer the world to work differently. If anything, geopolitics seems to be moving the other way. We are falling back on an older language of power, leverage and spheres of influence, as though outdated instincts from yesterday still provide the only credible map for the future. Perhaps that is partly a response to a more dangerous world, but it also reflects a failure of imagination. Periods of genuine change ought to give political leaders some licence to describe a different future, one shaped a little more by collaboration, equality, fairness and kindness, and a little less by the assumption that progress only counts if somebody else loses.

That will inevitably sound to some like technocratic naivety. Perhaps it is. But returning to repackaged versions of “might is right” is neither more imaginative nor, frankly, a particularly hopeful answer to the future. We have occasionally managed to decide that some things are bigger than the usual rules of competition, and science has often been at its best when that happens. AI may give us a reason to try again.

None of this requires us to stop research into Artificial General Intelligence. I doubt that is realistic, and I am not sure it would be desirable anyway. AGI could become one of the greatest scientific tools we have ever created. But there is an important distinction between intelligence and autonomy. An AI capable of designing a new cancer treatment does not automatically need permission to acquire resources, operate freely across the internet, rewrite itself or pursue open-ended objectives without human oversight. We can be extremely ambitious about what these systems can understand and still be cautious about how much freedom we give them to act.

For all the progress, we still do not really know how increasingly capable and autonomous systems will behave in every circumstance. Questions about reliability, control, unintended objectives and human oversight remain unresolved. That is not an argument against progress it is a reason to think more carefully about which forms of progress deserve to move fastest. If we can direct powerful systems towards bounded, valuable problems while we work through the harder questions of safety and control, then we gain time without standing still. We could be using that time to discover new medicines, improve energy technology and solve problems that matter now.

There is also the question of what we choose to reward. Enormous amounts of money are flowing into AI because the commercial prize for building increasingly capable general systems is equally enormous. That is understandable, but money has a habit of shaping not just which technologies are built, but which problems are considered worth solving in the first place. Imagine if even a meaningful proportion of that effort were judged against a different scorecard: how many cancers became treatable, how many years of healthy life were added, whether we discovered a new generation of antibiotics, made fusion practical, reduced the cost of clean energy or made measurable progress on climate change.

Those are not lesser ambitions. We may be creating the most powerful problem solving technology humanity has ever possessed at exactly the point when we have no shortage of problems worth solving. Competition will not disappear, and countries are not suddenly going to become altruistic. They do not need to. The great powers can still compete, AI companies can still build extraordinary businesses and researchers can still push towards AGI. But alongside that race, we could choose to run another one: to make pancreatic cancer survivable, to crack Alzheimer’s, to produce abundant clean energy, to develop antibiotics faster than bacteria can evolve resistance.

There may be a useful bargain hidden in that choice. We continue advancing AI and benefiting from what it can already do, but we give ourselves more time to decide how much autonomy its most powerful successors should have. If AI really is going to change everything, there would be something rather reassuring about some of its first great achievements being things that almost everyone, regardless of politics or ideology, could simply recognise as good.

The AI Agent Revolution

Phase 3 Ai is nearly upon us, make sure that you’re ready

Upon its launch back in 2022, Generative Ai was heralded as a game changer for our way of lives. Whilst there have been some astonishing advances in ai based human level conversation, document understanding and video and image creation, I would argue that Generative Ai is yet to have the ubiquitous, transformational effect across the business landscape that was predicted by some.

Now, 2 years later I find that whilst many organizations are experimenting with or deploying Phase 1 Generative Ai solutions, there is already ‘Ai fatigue’. I often hear ‘is Ai just another hype cycle?’.

The problem here is that whilst Conversational Generative Ai (Phase 1) and Reasoning Generative Ai (Phase 2) have been ‘very impressive’ without necessarily being transformative, I suspect that Phase 3 Ai, ‘Autonomous Agents’ could bring potential for significantly more impact.

This coming 3rd phase of AI isn’t just about chatting, it’s very much about doing. Could the far reaching consequences predicted in papers like gpt’s are gpt’s be very nearly upon us?

Across the industry Tech giants are ‘quietly’ racing to deploy AI agents that can autonomously execute complex business tasks.

On Nov 04th 2024 Microsoft unveiled Magentic-One, a sophisticated 3rd phase multi agent based AI system that doesn’t just analyze tasks, it also completes them. In benchmark testing, Magentic-One demonstrated the ability to autonomously navigate web interfaces, execute complex code, and manage multi-step business processes. Most notably, Magentic-One achieved performance comparable to specialized systems on industry-standard GAIA and AssistantBench tests, showcasing its potential for real-world applications. It’s important to say that Magentic-One uses GPT4o by default but is model agnostic, allowing data scientists to deploy the most appropriate LLM for specific a task.

Similarly Anthropic’s Claude 3.5 Sonnet launched in June 2024, represents another 3rd phase leap forward in autonomous AI capabilities. Whilst still in experimental stage, Claude 3.5 Sonnet can directly interact with computer interfaces, manipulate cursors, click buttons, and crucially correct its own mistakes in real-time. This self-correction capability, combined with its ability to execute complex sequences of actions, points to a future where Phase 3 AI systems might handle increasingly sophisticated business workflows.

Meanwhile, industry observers report that OpenAI is developing its own Ai Phase 3 agent-based platform dubbed ‘Operator’, with speculation about capabilities that could further advance the field. We could see this launched as early as Q1 2025. While these reports remain unconfirmed, they reflect the industry’s clear direction toward more 3rd phase autonomous AI systems.

The documented capabilities of these systems are remarkable.

Microsoft’s Magentic-One has demonstrated:

Autonomous web navigation and data collection
Complex code execution and debugging
Multi-step task planning and execution
Error recovery and task adaptation

These aren’t theoretical achievements. Microsoft’s research shows Magentic-One successfully completing complex tasks that previously required significant human intervention, including:

Multi-step data analysis workflows
Software development tasks
Document processing and analysis
Web-based research and compilation

But here’s the wake-up call: if you’re waiting to see how this plays out, you’re already late. Forward-thinking companies are already exploring how to integrate these capabilities into their operations. They’re rethinking workflows and reimagining what’s possible.

The risks of complacency are clear. Microsoft’s research indicates that agent-based systems can significantly reduce the time and effort required for complex tasks.

As we start to see 3rd Phase Ai systems become production ready, the gap between early adopters and laggards will widen very quickly. The brightness of the board room spotlight is likely to refocus on CTO’s and CDO’s who will be expected to answer questions about ‘Productivity’ and ‘Efficiency’.

The message for tech executives is clear: if you’re not already preparing for Ai Phase 3, start now.

This means:

Auditing current processes for agent automation potential
Building internal expertise in agent-based systems
Developing governance frameworks for Phase 3 Autonomous AI Agents and determining where and how to place humans most effectively in the loop
Creating pilot programs to test and learn

As always with Ai there are many oversight, ethical and production deployment considerations to consider with Phase 3 agent based AI. There are risks of overreach, transparency and accountability emphasizing the need for proper governance and control. Organizations must balance automation capabilities with appropriate safety measures.

Phase 3 Ai is coming and it’s coming soon.

We’re entering an era where AI isn’t just a ‘tool’ as it was in Phase 1 – the Ai space is moving forwards very quickly, Phase 3 agents will moves us more towards autonomous workforces, albeit (IMO) ones that will invariably still require humans in the loop. The companies that recognize this shift and act decisively will define the next generation of business leaders. Those that don’t risk becoming cautionary tales. The question isn’t whether AI agents will transform business operations, but rather which organizations will be ready when they do.