Tag: philosophy

What happens when technology moves faster than society can organise?

A child working in a Lancashire cotton mill in 1800 would have understood something about technological progress that we prefer to forget. The machines were extraordinary. They could produce cloth at a speed and price that would previously have seemed impossible, creating fortunes, transforming cities and helping Britain become the dominant industrial power of the nineteenth century. They also created workplaces in which children under ten worked appalling hours around dangerous machinery. Society eventually responded, but slowly. Britain passed its first factory legislation in 1802. More Acts followed. By 1833 there were factory inspectors. Employers found ways around the rules, so Parliament legislated again in 1844, 1847 and afterwards. The technology arrived first, prosperity followed, damage accumulated, and regulation came later. Enforcement then had to catch up with the regulation.

We have been repeating variations of this sequence ever since, although what has changed is the scale of the consequences. For most of human history, our capacity to cause trouble was reassuringly local. A badly governed kingdom could destroy itself. An army could sack a city. A new weapon could alter the balance between neighbouring states. Human stupidity was frequently catastrophic for the humans immediately concerned, but the planet was largely indifferent.

Industrialisation began changing that equation because it gave us something humans had never possessed before: the ability to amplify physical power independently of human muscle. Coal became steam, steam became industry, and industry led eventually to electricity, chemicals, mechanised warfare, mass production, aviation, nuclear physics, computing, biotechnology and artificial intelligence. Each step increased what a relatively small number of people could do. Our ability to organise ourselves did not increase at anything like the same rate, and in these times where Ai is upon us, I have started to wonder whether that may eventually be the fact that matters most about us.

We tend to imagine existential risk as a story about dangerous technologies: nuclear weapons, engineered pathogens, artificial intelligence, climate change. But perhaps the technology is not the underlying problem. The main issue is the widening gap between the rate at which humans acquire new capabilities and the rate at which humans can collectively decide how those capabilities should be used.

Technology compounds while institutions deliberate. A laboratory can discover something in an afternoon. A company can deploy software in minutes. Capital can cross the planet in milliseconds. An election still takes years. An international treaty can take a decade, with enforcement taking longer again. By the time the regulatory machinery has understood one generation of technology, the technology itself may already have moved on.

There is an obvious objection to this pessimism, which is that humans are capable of extraordinary cooperation. They plainly are.

Smallpox is probably the finest example. The United States and Soviet Union managed to cooperate on its eradication during the Cold War, despite being otherwise occupied with the possibility of annihilating one another. The World Health Organization coordinated surveillance and vaccination across borders and political systems. The last naturally occurring case was recorded in 1977, and in 1980 the WHO declared smallpox eradicated. It remains the only human infectious disease we have eliminated entirely.

The Montreal Protocol is another awkward example for anyone inclined towards despair, because it worked too. Scientists identified that chlorofluorocarbons were destroying stratospheric ozone. Governments negotiated restrictions. Industry developed alternatives. Nations complied sufficiently well that, assuming continued adherence to the Montreal Protocol, the ozone layer is now expected to return to roughly 1980 levels around 2040 across most of the world, around 2045 in the Arctic and around 2066 over Antarctica.

These are magnificent achievements, but the more I think about them, the less reassuring they become. Smallpox was unusually cooperative. Everyone wanted it dead. Eradicating it did not make America poorer while making Russia richer. There was no powerful smallpox industry warning about the economic consequences of vaccination, and no country gained a strategic advantage from allowing the disease to circulate for another twenty years. The virus had no shareholders.

Ozone depletion was more economically complicated, but it still had features that made cooperation possible. The causal chain was reasonably clear, a comparatively narrow family of chemicals was responsible, and substitutes could be developed. Civilisation did not depend upon continuously increasing its consumption of CFCs.

Climate change is rather different because the problematic technology is, in effect, modern civilisation itself. Cheap, abundant energy sits underneath almost everything we value: food, transport, housing, medicine, manufacturing, computing and economic growth. Countries are therefore being asked to constrain one of the foundations of prosperity today in return for benefits that are spread across the entire planet and extend decades into the future.

We have built an impressive global architecture of conferences, scientific assessments, treaties, targets and national commitments around the problem, and some of it has worked. Renewable energy has expanded rapidly, technologies have improved and many economies use energy more efficiently than they once did. Yet global energy demand continues to rise.

That should not really surprise us. The incentive to cooperate exists at the level of humanity, while the incentive to defect exists at the level where decisions are actually made. A government may believe deeply in reducing global emissions while also knowing that electricity prices, industrial employment and economic growth influence whether it survives the next election. A developing country may accept the science while reasonably asking why it should abandon the cheap energy that richer countries used to become rich. An oil producer may understand climate models perfectly well while knowing that another producer will happily sell the barrel it leaves underground.

Everyone can believe essentially the same thing and still behave differently because they occupy different positions inside the system.

Nuclear weapons offer an even starker case. For more than eighty years humanity has possessed machines capable of destroying civilisation and has somehow refrained from using them in war since 1945. There have been treaties, hotlines, inspections, arms-control regimes and painstaking diplomatic arrangements. At various points the United States and Soviet Union dismantled enormous numbers of weapons. This is strategic collaboration at its most impressive.

And yet thousands of nuclear warheads still exist. Nuclear powers are modernising their arsenals, and the international arms-control framework has been weakening.

There is something almost absurd about this. We discovered a technology that might end organised human life, correctly identified the danger, built international institutions specifically to manage it, negotiated agreements between bitter enemies and spent generations discussing disarmament. After all that, the rational conclusion of nearly every nuclear state is still that it would prefer to possess nuclear weapons while everybody else gets rid of theirs.

At the level of humanity, abolition makes obvious sense. At the level of an individual state, retention can make equally obvious sense. The difficulty is that states make decisions and humanity does not.

Artificial intelligence may become the purest version of this problem because technological development and competitive pressure are now so closely fused. There is already substantial international cooperation around AI safety. Governments have held summits, research institutes have been created, and leading AI companies have made commitments around testing, evaluation and risk thresholds. There is serious work going on to understand the possibility of misuse, loss of control and wider social harm.

The difficult question is what happens when safety and strategic advantage point in opposite directions.

Imagine that one AI laboratory believes the next generation of systems requires another year of safety research, while its competitor does not. Or America pauses and China continues. Or China pauses and America continues. Or both governments agree to slow down, only for a third state to decide that an eighteen-month technological lead is the greatest strategic opportunity it is likely to see in a century.

Every participant may understand the systemic risk perfectly. It may not matter. The rewards for being first are concentrated, while the risks of catastrophe are distributed. The structure almost demands the race.

We often blame this sort of behaviour on greed, but greed is too simple an explanation. Most people participating in the system are not unusually greedy. They are responding quite rationally to the incentives around them. A chief executive who voluntarily makes their company permanently less competitive in pursuit of an abstract global good may eventually be replaced by one who will not. A fund manager who accepts consistently lower returns because humanity would benefit from less economic activity loses assets to a fund manager who does not. A politician who deliberately reduces national prosperity while neighbouring countries continue growing will eventually meet an electorate. An employee who rejects economic competition altogether still needs somewhere to live.

Even our retirements depend on the machine continuing to run. Pension funds own equities. Equities require returns. Returns require businesses to produce profits and, usually, to grow them. Businesses compete for customers, labour, resources and capital. Governments depend on economic activity for tax revenues, which support healthcare, education, pensions, defence and the very institutions that are supposed to protect us from the consequences of all this activity.

We have made ourselves economic objects inside the system we created. There is no obvious lever marked stop.

Capitalism is particularly effective at accelerating useful technology precisely because it converts human ambition into competition. Thousands of organisations pursue thousands of ideas simultaneously. Failure kills the weak ones. Success attracts capital. Successful technology spreads. It is an astonishing discovery engine. What I am much less certain about is whether it is a good survival engine.

The same competitive process that gives us better batteries, medicines and computers makes restraint unstable. If an opportunity exists, somebody has an incentive to exploit it. If one country prohibits something, another can permit it. If one company decides a capability is too dangerous to commercialise, another can decide that the danger is manageable. Technological civilisation therefore contains something close to an evolutionary process. Possible capabilities occupy niches, economic actors explore them, and profitable capabilities reproduce.

Nobody needs to be in charge, which is perhaps the most unsettling part. We tend to imagine that avoiding catastrophe requires humanity to make a catastrophically bad decision, but perhaps it requires nothing of the sort. Billions of individually sensible decisions may be enough.

This becomes uglier as resources tighten. Our instinct when something valuable becomes scarce is not universal generosity but resource guarding. Individuals guard property. Companies guard intellectual property and markets. Nations guard energy, food, water, semiconductors, strategic minerals, territory and military capability.

There is perfectly good evolutionary logic behind this. A tribe that shared its winter food with every neighbouring tribe might have been morally admirable for a surprisingly short period of time. But behaviours that protected small human groups can become pathological when eight billion people operate inside a technologically integrated civilisation.

Climate instability threatens food or water supplies, so nations secure their own. Advanced AI becomes strategically important, so nations restrict chips and accumulate compute. Energy becomes uncertain, so governments secure domestic supply. Military threats increase, so defence spending rises. Other countries observe this behaviour and respond in kind.

The rational response to insecurity creates more insecurity. We call it deterrence, resilience, national security, strategic autonomy or supply-chain protection depending on which ministry is writing the document. From a sufficient distance it can look remarkably like frightened apes piling resources in their corner of the enclosure.

The hopeful answer is that institutions can evolve. They can, and they always have. The trouble is that biological evolution gave us hundreds of thousands of years to develop cooperation. Political institutions have had thousands. Industrial civilisation has had roughly two centuries. Nuclear weapons have existed for eighty years. Modern computing for less. Frontier AI changes noticeably between budget cycles.

The intervals are shrinking.

Perhaps humanity will repeatedly rise to the occasion. We may build stronger institutions, negotiate better treaties, regulate dangerous technologies, manage climate change, control synthetic biology, prevent nuclear war and somehow develop artificial intelligence more powerful than ourselves without losing control of it. None of those outcomes is impossible.

The problem is that we may have to succeed too many times.

Existential risk is asymmetric in a way ordinary politics is not. Humanity can successfully manage a dangerous technology for a century and receive no special prize. It merely gets another century. Failure may need to happen only once.

Nuclear deterrence has to keep working. Pandemic controls have to catch the engineered pathogen that matters. AI safety has to remain adequate through every major capability transition. Climate cooperation has to endure across elections, wars, recessions and geopolitical realignments. We are effectively taking an examination in which the questions become harder each year and one sufficiently bad answer ends the examination.

That is why the achievements of smallpox eradication and the Montreal Protocol offer less comfort than they first appear to. They prove that humans can organise brilliantly. They do not prove that we can organise brilliantly every time, everywhere, indefinitely, and under conditions where enormous wealth or strategic power rewards the first person who refuses to cooperate.

That is increasingly the standard technological civilisation appears to demand of us.

Perhaps the final mistake is imagining that extinction would have to arrive dramatically. There may never be a single moment when humanity gathers around a table and chooses destruction. There may only be another investment decision, another election, another weapons programme, another model release, another mine, another power station, another perfectly defensible decision to protect national interests.

Each participant will have reasons, and most of them will probably be good ones. The strange creature we have created from markets, states, corporations, technology and eight billion individual lives will simply continue doing what the system encourages it to do: compete, expand, optimise and protect its resources.

Technological power will keep increasing because somebody benefits from increasing it. Collective restraint will continue to move more slowly because restraint only works when enough other people agree to exercise it too.

Eventually, our ability to create dangerous capabilities may outpace our ability to understand, regulate or contain them. For a while, that gap may be survivable. We may make mistakes, absorb the damage, learn and tighten the rules. But as the technologies become more powerful, the margin for error narrows. At some point, a failure may no longer be something civilisation can correct after the fact. The first time our capacity to cause harm exceeds our capacity to recover from it could also be the last time we get to learn from the mistake.

If that happens, human extinction will not prove that we lacked intelligence. Intelligence may turn out to be the thing we possessed in dangerous abundance. Our failure will have been organisational. We became extraordinarily good, extraordinarily quickly, at discovering what could be done, but never became equally good at agreeing what should not be done.

There is no obvious reason to believe the universe will give us unlimited attempts to learn the difference.

The Eight Hours We Haven’t Planned For

In 1930 John Maynard Keynes made one of the most famous predictions in economics. Technological progress, he thought, would make his grandchildren so productive that they might need to work only 15 hours a week.

The prediction is usually remembered because it was wrong. People became vastly richer, but they continued to work.

Less often remembered is that Keynes was not entirely relaxed about the alternative. If technology eventually solved what he called the economic problem, humanity would encounter another one: what to do with its freedom. People had been trained for generations to strive, he argued, and might prove surprisingly bad at doing anything else. Those with money but few duties or ties did not strike him as particularly encouraging examples of life after work.

Two years later, in an Austrian village called Marienthal, researchers stumbled across a darker version of the same problem.

The village’s textile mill had closed, leaving most families dependent on unemployment support. Poverty obviously dominated what followed, which makes Marienthal a poor model for an affluent automated future. But something else happened that fascinated the researchers. Time itself seemed to change.

The unemployed men walked more slowly. Many stopped repeatedly while crossing the village. Most no longer wore watches. Library borrowing fell rather than rose. Presented with an abundance of time, people did not automatically fill it with reading, hobbies or civic life. The structure that had made time useful had disappeared.

Nearly a century later, artificial intelligence raises a much larger version of Keynes’s question.

Most discussion about AI and employment is understandably economic. Which jobs disappear? How quickly does productivity rise? Who owns the machines? How should the income they generate be distributed?

But suppose, eventually, that those problems can be solved. Imagine a society rich enough to provide people with comfortable lives even when relatively little human labour is required.

What will everyone actually do all day?

For most adults, work occupies roughly eight hours of every weekday. For children and teenagers, school does much the same. Add commuting, preparation, homework and the mental space occupied by both, and a remarkable proportion of human existence is organised around the requirement to be economically useful.

Now imagine that requirement begins to disappear.

Not because useful things no longer need doing, but because machines can do more of them, more cheaply and eventually better. They diagnose illnesses, write software, analyse businesses, prepare contracts, design buildings and teach mathematics. If robotics makes comparable progress, much physical labour may follow.

Intelligence becomes a utility. Eventually labour may begin to resemble one too.

The economic consequences would be enormous. The psychological consequences may be stranger.

Work does much more than produce income. It tells people when to wake up. It supplies colleagues, deadlines, routines and mild competition. It creates achievements and disappointments. It provides status and identity. It gives someone a reason to leave the house and, perhaps most importantly, gives other people reasons to expect them to arrive somewhere.

School performs a similar function for the young. It is supposedly an institution for transferring knowledge, but it is also a machine for structuring childhood. It puts several hundred young people in the same place at the same time. It forces them to coexist, encounter people they did not choose, play sport, sit examinations, perform plays, make friends, lose arguments, cope with boredom and occasionally discover that they are good at something they had never considered trying.

Remove much of work and school and society does not merely lose two institutions.

It loses much of its timetable.

Retirement gives us the closest existing approximation. Someone reaches sixty or sixty-five and, often quite abruptly, discovers that the institution which has organised much of adult life has disappeared.

Some flourish. They have family, friendships, travel, voluntary work, sport and projects waiting for them. Others discover that work had quietly been supplying much of their social world and sense of identity.

But a person retiring at sixty five has had four decades to work out who they are. They have accumulated relationships, interests and obligations. With luck, they know what gives shape to a Tuesday when nobody is paying them to be somewhere at nine o’clock.

An eighteen year old has not.

That may be the harder problem.

If an eighteen year old enters a world in which employment is optional, scarce or simply unnecessary, what exactly are they supposed to do with the next seventy years?

At that age most people have barely begun to discover what they care about. School, university and work provide tracks to follow while identity gradually forms. They impose sequence. Study this. Qualify. Find work. Become better at something. Change direction. Perhaps raise a family. Eventually retire.

Remove the tracks and society may give young people unprecedented freedom before they have developed the judgement required to use it.

The immediate alternatives are not wholly reassuring.

An enormous industry already exists whose purpose is to capture unclaimed attention. Social networks, games, streaming services and personalised entertainment compete continuously for spare minutes. If work releases another eight hours a day, some of the richest companies in history will have every incentive to occupy them.

The technologies will also become better at the job. AI will be able to create endless games, stories, relationships and virtual worlds tuned precisely to the psychology of one individual. Boredom may become technologically obsolete.

A society could therefore achieve something close to material abundance while millions of people spend much of their lives inside experiences optimised to stop them leaving.

Yet boredom may not be the deepest difficulty.

Decades of research into unemployment suggest that employment supplies several things besides pay: structure to time, social contact, status, regular activity and participation in a collective purpose. Marie Jahoda, one of the researchers at Marienthal, later called these the latent functions of work. Modern research still finds links between these functions and mental wellbeing.

But there may be something underneath all of them.

People want to matter.

Psychologists use that word in a fairly precise way. Mattering means not simply believing that your life has purpose, but feeling significant to other people. One component is dependence: the knowledge that somebody relies on you. Another is the feeling that your actions genuinely make a difference.

This suggests that the usual discussion about the end of work may be asking the wrong question.

We ask what people will do.

Perhaps we should ask who will need them.

There is an important difference.

I can decide that learning Portuguese is my purpose. I can spend five years becoming an excellent pilot. I can go to the gym every other day, ride horses or study astronomy. All could make my life richer.

But if I decide not to do any of those things tomorrow, nobody necessarily suffers.

Work is different because it often makes our actions consequential to other people. A patient expects the doctor. A pupil expects the teacher. A customer expects the plumber. Your colleague needs the report because she cannot finish something until you do.

Even an ordinary job embeds a person in a small network of dependence.

The uncomfortable possibility is that AI does not merely make us less necessary to the economy. It makes us less necessary to one another.

If an AI can teach the child, diagnose the patient, repair the machine, give the legal advice, drive the passenger and perhaps eventually provide companionship, many of the practical reasons humans have historically had to depend upon other humans begin to weaken.

Materially, this could be wonderful.

Socially, it presents a peculiar problem.

A world in which nobody needs to do very much for anyone else may be comfortable while also making it harder to answer one of the oldest human questions: where would there be a hole if I disappeared?

AI could even simulate the answer. An artificial companion could tell you that it missed you. A game could give you a vital mission. A virtual organisation could provide status, promotion, difficulty and praise.

But there is a difference between being told that you matter and knowing that something outside yourself actually depended upon your turning up.

That distinction may become increasingly valuable.

The same is true of time.

Work and school do something so familiar that we barely notice it: they synchronise millions of lives.

Monday morning means roughly the same thing to everybody. Evenings are when many people stop working. Children have school holidays together. Saturday afternoon works as a social institution because large numbers of people are available at the same time.

Research into time use shows that this synchronisation matters. Couples actively coordinate working hours to create shared leisure, and people whose routines are badly out of step with society often pay a social cost.

This leads to a curious possibility.

A post work society could have an abundance of individual free time and a shortage of common time.

If everybody is free whenever they like, everybody may choose differently. The rhythms that currently coordinate us could weaken along with the jobs that created them.

Saturday means something partly because Monday exists.

This is why simply building more leisure facilities does not quite solve the problem.

A future rich in sports centres, parks, workshops, libraries and virtual worlds might offer people an extraordinary range of activities. But activities are not the same as obligations, and private leisure is not the same as shared life.

The institutions that matter most may therefore be those that create two things work currently supplies almost accidentally: mutual dependence and common time.

A football team does this. Training is on Wednesday because everyone has agreed that Wednesday is when they train. If the goalkeeper does not appear on Saturday, ten other people notice.

An orchestra does it. So does a theatre company, a voluntary organisation, a scientific project, a community garden, a care network or a group restoring a woodland. Their value is not simply that they keep people busy. They create roles in which other people rely on you.

Perhaps that is the infrastructure a post-work society would actually need.

Not organised leisure, exactly. Something closer to organised significance.

It would need institutions capable of creating what might be called voluntary necessity: commitments freely chosen, but real enough that once chosen they impose duties.

There is a paradox here. We imagine freedom as the removal of obligations. A society liberated from compulsory work might discover that a satisfying life requires putting some obligations back.

The difference is that people could choose which ones.

Instead of being compelled by economic necessity to spend forty hours a week doing whatever the labour market happens to reward, people might have much greater freedom to choose the communities, skills, causes and people to whom they wish to become useful.

That could be one of AI’s more attractive consequences.

It might allow societies to distinguish for the first time at scale between what is economically necessary and what is humanly necessary.

Looking after an elderly neighbour may generate little GDP. Raising children, maintaining a community garden, mentoring a teenager or helping run a local charity can be economically marginal while being deeply consequential to the people involved.

Our present economy often gets this backwards. Some of the activities people consider socially valuable are poorly paid, while some highly paid work leaves those doing it wondering whether anything would change if the job disappeared.

AI could deepen that absurdity, or help us escape it.

Education would then have to change too.

If an infinitely patient AI tutor can teach algebra, chemistry or French, schools may need to spend less time transmitting information and more time teaching young people how to participate in human institutions.

How do you become someone others can rely upon? How do you cooperate, lead, lose, persist, make friends, resolve conflict and keep a commitment after the enthusiasm that produced it has disappeared?

These are not secondary skills in a post-work society. They may become the central ones.

A good school might still teach history and mathematics, but its deeper task would be to help a young person discover not merely what interests them but where they can become useful to others.

That is harder than teaching algebra. Algebra has a syllabus.

A life does not.

There is good reason nevertheless to be optimistic.

Human beings have never shown much enthusiasm for doing nothing. When released from necessity they repeatedly invent difficulty. They run marathons, climb mountains, become obsessive gardeners, learn musical instruments, compete at sport and spend years mastering skills with almost no economic value.

The problem is not that humans lack the desire to strive.

It is that the market has spent several centuries performing the useful function of connecting that striving to a social structure.

The next task may be to make those connections without requiring a wage.

That is why retirement is an incomplete model for the future. A sixty five year old often enters retirement already embedded in a lifetime of relationships and obligations. There may be children, grandchildren, old colleagues, clubs, neighbours and interests accumulated over decades. Work disappears, but the person does not start again from nothing.

An eighteen year old may.

So perhaps the most important question raised by a post-work society is not how an eighteen-year-old fills seventy years of leisure.

It is how, over those seventy years, they become entangled in the lives of other people.

For most of human history necessity did much of that work for us. Families depended upon one another because survival demanded it. Communities depended upon cooperation. Employers needed workers. Workers needed wages. Scarcity forced humans into relationships of mutual reliance whether they liked it or not.

AI may loosen some of those bonds by solving the scarcity that created them.

That should still be welcomed. There is nothing intrinsically noble about drudgery, poverty or having no choice about how to earn a living.

But removing economic necessity does not remove the human need to be consequential.

If anything, it makes that need more visible.

The deepest challenge of the AI age may therefore turn out not to be finding enough things for humans to do. Machines and entertainment systems can provide endless things to do.

The harder task will be constructing a society in which people still have reasons to need one another.

For most of modern history we have asked young people: what can you learn to do that somebody will pay you for?

An eighteen year old in an age of abundant intelligence may face a much larger question.

Who will you choose to become useful to?

A good society would make sure there are many answers.

What happens when intelligence becomes a utility?

A friend and I were talking about AI on the bus to last weekend’s football game and, as usual, ended up on the familiar questions. How good are these systems getting? Which jobs will go first? What happens if they eventually become better than us at most things?

As the bus wound its way through the inevitable traffic outside the stadium, we got onto a different question. If the cost of producing useful intelligence keeps falling, what happens when it becomes so cheap and readily available that we begin to treat it like a utility?

For most of human history, useful intelligence has been expensive. Diagnosing an illness, interpreting a contract, writing software or analysing a company usually requires somebody who has spent years learning how to do it. Societies responded by building institutions around that scarcity. Universities produced expertise, professions certified it and companies assembled it.

When you pay a good lawyer or engineer, you are not really buying an hour. You are buying a small slice of everything that lawyer or engineer has learned to make that hour useful.

The cost of intelligence is already falling quickly. When GPT-4 appeared in 2023, OpenAI charged $30 per million input tokens and $60 per million output tokens. More capable models now cost a fraction of that. The comparison is not perfectly like for like, but the direction is clear enough. More intelligence is being delivered for much less money.

Electricity offers a useful precedent. Before electrification, factories had to produce power close to where it was consumed. Electricity changed that. Once power became reliable and cheap, firms stopped designing around its scarcity and started assuming it would simply be there.

The important effect was not cheaper machinery. It was what followed. Elevators made tall buildings practical. Refrigeration transformed food distribution. Washing machines changed domestic life. Computing eventually grew on top of the same infrastructure. Once power was abundant, whole categories of activity that had previously made little sense became viable.

AI may do something similar to cognition.

There is a great deal of thinking that simply does not happen today because it costs more than it is worth. Companies do not investigate every small opportunity. Doctors cannot ask ten specialists to review every patient. Scientists cannot explore every plausible hypothesis. These limits are so familiar that we barely notice them.

However, lower the cost far enough and some of those limits start to disappear. Imagine every medical diagnosis being looked at from several directions. One system checks drug interactions. Another looks for unusual conditions. Another scans the latest research. A fourth is told to challenge the conclusions of the others.

That does not necessarily make the doctor redundant. It makes something that is currently exceptional, several expert opinions on one case, routine.

The same logic extends much further. A small business might have something resembling a lawyer, finance team, developer and market analyst available whenever it needs them. A student could keep asking for a different explanation until a difficult idea finally clicks. A scientist might explore a hundred possibilities before deciding which are worth testing in the real world.

This is why it is important not to get trapped solely in the immediate questions about AI: which jobs disappear, which tasks are automated, which professions are disrupted. We also need to think about what a post-AI economy looks like once intelligence is cheap and widely available.

History suggests that when one thing becomes plentiful, something else becomes scarce.

The internet gave us almost unlimited information and left us fighting for attention. Digital cameras made photographs virtually free without making everyone a good photographer.

If AI gives us almost unlimited analysis, then another analysis may not be worth very much.

A management team may soon be able to generate a hundred plausible strategies before breakfast. The difficult part will still be choosing one and living with the consequences.

The same is true of financial analysis. If models and reports can be produced almost endlessly, value shifts towards people who can explain what the numbers mean, spot dubious assumptions, understand the business behind them and help somebody decide what to do next. Good analysts may spend less time calculating and more time interpreting.

Producing an answer is also not the same as knowing that it is right. AI might produce a legal opinion in seconds, while somebody still has to decide whether they trust it enough to act on it. Lines of software can be generated quickly, but proving that the result is safe enough to run a bank is a very different problem.

So I do not think expertise simply disappears. It changes shape. Knowledge and analytical skill matter, but so do experience, credibility, judgement and responsibility. AI can make the first two much cheaper. The others are more stubborn.

This could alter the shape of companies too. Organisations exist partly because useful knowledge is scattered across people and coordinating those people is costly. If AI reduces the cost of both expertise and coordination, some activities that currently require a large organisation may no longer do so.

A small company may gain access to capabilities that once required entire departments. Large organisations will still enjoy advantages in capital, reputation, distribution, proprietary data and physical assets. But the ability to employ large numbers of clever people may no longer confer quite the same edge.

By the time my friend and I reached the football, fortunately, we had stopped talking about AI. Like it or not, football is already drenched in analysis: expected goals, tracking data, tactical models, performance statistics. AI will provide vastly more of it.

Yet it is hard to imagine that wealth of data making the match itself more meaningful for the fan. There is still only one game taking place. Nobody knows exactly what will happen. If your team misses a penalty, a computer can simulate a thousand versions in which it goes in, but none alters the afternoon you actually experienced.

Perhaps that hints at another consequence of abundance. Some things remain stubbornly scarce.

And this is where the electricity comparison breaks down.

Electricity is passive. It can power a television but does not tell you what to watch. Intelligence can. An AI helping with a decision can also choose which evidence to show, how to frame the question and which possibilities appear sensible.

That makes intelligence a peculiar sort of utility. We may come to depend on it while it is also helping to shape the way we understand the world.

Questions that sound technical today may therefore become important. Who owns these systems? What are they optimising for? How independent are two supposedly different opinions if both depend on the same underlying models? Who takes responsibility when everyone follows the answer because everyone assumes somebody else has checked it?

Many of our institutions were built for a world in which intelligence was difficult to find and expensive to organise. Schools, professions, companies and bureaucracies all reflect that scarcity in different ways.

AI will not sweep them away. But if competent intelligence becomes something we can summon almost without thinking about the cost, it would be surprising if they all continued to look much the same.

We spend an enormous amount of time debating how intelligent AI will become. I think we need to spend much more time thinking about the world that follows if intelligence becomes cheap and abundant.

Technology can change very quickly. Institutions, education, companies and social norms usually do not. If intelligence really is becoming a utility, the danger is not simply that we underestimate what AI can do. It is that the technology arrives long before we have worked out what we want to do with it.

We’re racing towards AGI. But what do we actually want AI to do?

The discussion about artificial intelligence felt rather different this week. Jack Clark, the Brighton-born co-founder of Anthropic, was talking about the possibility that we may eventually need to slow the development of the most powerful AI systems, and even about independently verifiable ways of shutting them down. Then, almost in counterpoint, Nature reported on a “virtual biotech” involving tens of thousands of AI agents working together on drug discovery and identifying a promising new approach to lung cancer treatment. One story was about restraint, the other about possibility. Put together, they seem to capture a choice we are beginning to face.

We spend a great deal of time talking about the race towards Artificial General Intelligence: which company will get there first, which country will dominate it, how much more capable the next model will be. There are perfectly understandable commercial and geopolitical reasons for this. But the race itself can start to dictate the objective. In trying to build an intelligence capable of doing almost everything, we may be paying too little attention to a simpler question: what do we actually want all this intelligence to do for us?

Cancer seems an obvious place to start. Rather than directing every new increment of computing power towards making one general purpose model slightly better at everything, imagine deliberately creating hundreds, perhaps eventually thousands, of specialised AI agents focused narrowly on understanding and treating it. Some could concentrate on individual cancers, others on genetics, protein interactions, immune response, drug discovery, pathology, imaging or clinical trials. They would work alongside scientists, doctors and laboratories, testing ideas against evidence, learning from failed experiments and sharing what they discover.

The important point is that this isn’t just theoretical. Large numbers of AI agents collaborating on scientific problems are already beginning to appear in real research; the Nature article is one early example. So extend the idea all the way to the frontier. Apply the same intensity to Alzheimer’s, antibiotic resistance, fusion energy, energy storage, food production, clean water and other stubborn problems humanity has been trying to solve for decades. The attraction is not simply that these are worthy causes. Narrowing the AI mission may also be a sensible way to use increasingly powerful AI while we learn how to make it safer.

That matters because we do not have to choose between racing ahead and stopping altogether. There is a middle course in which we continue to improve the intelligence, and continue putting it to work on difficult real world problems, while being more cautious about the autonomy we give increasingly general systems. Capability and permission do not have to advance at the same speed. A system can become dramatically better at science, medicine or engineering without automatically being given greater freedom to acquire resources, rewrite itself or act independently across the world. Narrowing the mission, at least for a time, could give us something that may soon become scarce: room to think.

There is a larger version of the idea too. What if the major powers agreed that a handful of these problems were simply too important to become another winner-takes-all technological race? The United States might lead a major international effort on cancer. China might commit enormous resources to fusion or energy storage. Europe could focus on neurodegenerative disease, with Britain, India, Japan and others taking responsibility for different challenges. The precise allocation is not the point. Each country would bring its best scientists, laboratories, computing power and AI systems to bear on a defined human problem, then share what it discovers.

Of course, this would be difficult. Countries compete, companies have shareholders, governments have security concerns and intellectual property has enormous value. None of that disappears because we might prefer the world to work differently. If anything, geopolitics seems to be moving the other way. We are falling back on an older language of power, leverage and spheres of influence, as though outdated instincts from yesterday still provide the only credible map for the future. Perhaps that is partly a response to a more dangerous world, but it also reflects a failure of imagination. Periods of genuine change ought to give political leaders some licence to describe a different future, one shaped a little more by collaboration, equality, fairness and kindness, and a little less by the assumption that progress only counts if somebody else loses.

That will inevitably sound to some like technocratic naivety. Perhaps it is. But returning to repackaged versions of “might is right” is neither more imaginative nor, frankly, a particularly hopeful answer to the future. We have occasionally managed to decide that some things are bigger than the usual rules of competition, and science has often been at its best when that happens. AI may give us a reason to try again.

None of this requires us to stop research into Artificial General Intelligence. I doubt that is realistic, and I am not sure it would be desirable anyway. AGI could become one of the greatest scientific tools we have ever created. But there is an important distinction between intelligence and autonomy. An AI capable of designing a new cancer treatment does not automatically need permission to acquire resources, operate freely across the internet, rewrite itself or pursue open-ended objectives without human oversight. We can be extremely ambitious about what these systems can understand and still be cautious about how much freedom we give them to act.

For all the progress, we still do not really know how increasingly capable and autonomous systems will behave in every circumstance. Questions about reliability, control, unintended objectives and human oversight remain unresolved. That is not an argument against progress it is a reason to think more carefully about which forms of progress deserve to move fastest. If we can direct powerful systems towards bounded, valuable problems while we work through the harder questions of safety and control, then we gain time without standing still. We could be using that time to discover new medicines, improve energy technology and solve problems that matter now.

There is also the question of what we choose to reward. Enormous amounts of money are flowing into AI because the commercial prize for building increasingly capable general systems is equally enormous. That is understandable, but money has a habit of shaping not just which technologies are built, but which problems are considered worth solving in the first place. Imagine if even a meaningful proportion of that effort were judged against a different scorecard: how many cancers became treatable, how many years of healthy life were added, whether we discovered a new generation of antibiotics, made fusion practical, reduced the cost of clean energy or made measurable progress on climate change.

Those are not lesser ambitions. We may be creating the most powerful problem solving technology humanity has ever possessed at exactly the point when we have no shortage of problems worth solving. Competition will not disappear, and countries are not suddenly going to become altruistic. They do not need to. The great powers can still compete, AI companies can still build extraordinary businesses and researchers can still push towards AGI. But alongside that race, we could choose to run another one: to make pancreatic cancer survivable, to crack Alzheimer’s, to produce abundant clean energy, to develop antibiotics faster than bacteria can evolve resistance.

There may be a useful bargain hidden in that choice. We continue advancing AI and benefiting from what it can already do, but we give ourselves more time to decide how much autonomy its most powerful successors should have. If AI really is going to change everything, there would be something rather reassuring about some of its first great achievements being things that almost everyone, regardless of politics or ideology, could simply recognise as good.