Category: AI Safety & Governance

China’s rise and the growing risks of the AI race

For the past three years, artificial intelligence has been dominated by a familiar collection of American companies. OpenAI, Anthropic, Google and Meta have attracted extraordinary investment, recruited some of the world’s best researchers and built increasingly powerful models. Their progress has been remarkable, although the cost of achieving it has been equally remarkable. Training a frontier model requires enormous computing resources, and operating one at scale creates another substantial bill.

China has approached the same problem from a different position. American restrictions on advanced semiconductor exports have made access to the most powerful AI chips more difficult, while fierce domestic competition has placed considerable pressure on prices. Chinese developers have consequently had strong incentives to find more efficient ways of training and running models. What began partly as a response to technological constraints has become a competitive advantage.

The arrival of DeepSeek R1 in January 2025 brought this into focus. Here was a Chinese company offering reasoning capabilities that challenged leading American systems while publishing its model weights and claiming substantially lower training costs. Some early comparisons were overstated, particularly those confusing the cost of an individual training run with the full expense of developing a frontier model. Nevertheless, DeepSeek demonstrated that sophisticated AI could be developed and distributed using a different economic model.

Alibaba, Moonshot AI, Z.ai and MiniMax have since helped establish a Chinese AI industry considerably broader than many Western observers appreciate.

How do Chinese and Western models compare?

Comparing AI models has become surprisingly difficult. Some excel at mathematical reasoning, others at writing software, analysing documents or operating tools autonomously. Performance also changes depending on how much computing power a model is allowed to spend answering a question.

The following comparison is deliberately approximate. Benchmark scores are indicative rather than definitive, and not every model has been evaluated under identical conditions. The pricing figures are similarly illustrative, reflecting reported API rates rather than a uniform, independently verified price survey.

Region	Company	Model	Principal strengths	Score*	API input / output**
US	OpenAI	GPT-6 Astra	Reasoning, coding, agents	53	$10 / $50
US	Anthropic	Claude Opus 5.5	Reasoning, coding, autonomous tasks	58	$4 / $20
US	Google	Gemini 4 Argon	Multimodal reasoning, long context	53	$2 / $10
US	Meta	Muse Spark 1.3	General reasoning, efficient inference	48	$1.25 / $4.25
France	Mistral	Mistral Large 4	Coding, enterprise deployment	N/V	$0.68 / $2.09
China	Xiaomi	MiMo-V2.6-Pro	Reasoning, coding	46	N/V
China	Alibaba	Qwen 3.8 Max	Coding, multimodal applications	45	$2 / $6
China	Z.ai	GLM 5.3	Coding, agents	45	Varies
China	DeepSeek	V4 Pro	Reasoning, coding	36	$0.66–1.74 / $1.98–3.48
China	Moonshot AI	Kimi K3	Long-context reasoning, agents	44	$0.50–3 / $12–15
China	MiniMax	M3	Coding, multimodal tasks	N/V	N/V

Approximate benchmark index points, not percentages or a uniform ranking. Indicative USD per million input/output tokens, October 2026. Prices vary by provider and usage conditions; some entries have not been independently verified. N/V means not verified.

What stands out is the number of Chinese developers competing across essentially the same technical territory as their American counterparts. Chinese models are increasingly designed for sophisticated reasoning, software development and autonomous agents, rather than simply providing inexpensive alternatives for routine tasks.

American proprietary models remain strong at the frontier, but the gap varies considerably by task. There is also an important distinction between performing well on a benchmark and operating reliably in a business environment. I’ve explored that problem in an earlier essay comparing leading AI models, where apparently small differences in performance can be much less meaningful than they first appear.

Pricing introduces another dimension. Several Chinese developers offer capable models at substantially lower prices than leading Western frontier systems, although Western companies also provide inexpensive smaller models. OpenAI’s GPT-6 Luna, for example, has been offered at $0.10 per million input tokens and $0.50 per million output tokens. Comparing prices without considering capability is rather like comparing the cost of two employees without asking what work they can actually do.

The effect of Chinese competition has therefore been to increase pressure across the price-performance spectrum. This becomes particularly important for AI agents that may make hundreds or thousands of model calls while completing a task. The wider economic consequences connect with my earlier essay, What is a company for when intelligence is cheap?

Another distinction concerns distribution. Several Chinese developers publish open-weight models that organisations can download and operate themselves, rather than accessing them exclusively through a commercial API. Meta and Mistral also offer open-weight models, but Chinese companies have made this approach particularly important to their competitive positioning. It offers flexibility and infrastructure independence, although it also makes subsequent control over model deployment considerably harder.

China’s industrial advantage

China’s progress extends beyond a collection of talented research laboratories. Artificial intelligence forms part of a broader industrial strategy intended to reduce dependence on foreign technology and strengthen domestic manufacturing and infrastructure.

The government’s AI Plus initiative encourages adoption across established industries, including manufacturing, transport and robotics. China already has enormous industrial capacity, sophisticated supply chains and companies operating at scale in electric vehicles, telecommunications and automation. These provide opportunities to apply AI to physical processes rather than concentrating primarily on software and information work.

An AI system that improves factory scheduling, reduces defects or enables an industrial robot to perform a new task has an economic value that can be measured relatively directly. It may be less glamorous than demonstrating that a model can solve an obscure mathematical problem, but the potential productivity gains are substantial.

There are limitations. Semiconductor restrictions remain a constraint, and government support can sustain investments that would struggle under ordinary commercial scrutiny. Nevertheless, the combination of domestic competition, industrial demand and strategic investment gives Chinese developers considerable opportunities to improve their products.

It also creates pressure to demonstrate progress, which introduces a less comfortable aspect of the competition.

The safety problem

The incentives driving AI development are not necessarily aligned with those required for safety. Releasing a more capable model ahead of a competitor can bring considerable commercial rewards. Delaying that release for additional evaluation may produce little visible benefit, particularly if a rival proceeds anyway. The company exercising caution risks losing customers and investment, while the benefits of its restraint are shared across the industry.

This is not exclusively a Chinese problem. American companies face similar pressures, although their approaches to testing, disclosure and regulation differ.

The concern becomes more serious as models acquire the ability to operate autonomously. A conventional chatbot produces an answer that someone can examine before acting upon it. An AI agent may write and execute code, access information systems or perform sequences of actions with limited human supervision. Errors that were previously confined to a conversation can become operational incidents. This connects with a question I explored in Who signs their name to the answer?, concerning responsibility when organisations delegate consequential decisions to AI.

Our ability to evaluate these risks remains immature. Developers can test models against known categories of harmful behaviour, but cannot comprehensively anticipate every interaction between a model, its tools and an unfamiliar environment.

A Reuters report published on 9 October 2026⁠ examined research into safety disclosures by major Chinese AI developers. The study found that only 31 of 857 model releases, approximately 3.6%, had publicly disclosed model-specific safety evaluations. Just nine had published those evaluations at or before launch.

These figures do not establish that the remaining models were released without testing, nor did the research provide an equivalent comparison with American developers. They do, however, illustrate how little independent evidence may accompany the release of increasingly capable systems.

China is not ignoring AI safety. Its government has introduced regulations and safety initiatives addressing generative AI services and emerging risks. However, regulatory priorities differ between countries. China’s approach places considerable emphasis on information control, social stability and compliance with national requirements, while Western frameworks have tended to give greater prominence to privacy, discrimination, transparency and systemic risks. There is overlap, but reaching agreement on what constitutes an acceptably safe model is difficult.

Open-weight models introduce another complication. Once distributed, their developers cannot reliably withdraw them, enforce safety updates or prevent others from modifying safeguards. There are substantial benefits to openness, including independent research and scrutiny, but these need to be considered alongside the difficulty of controlling subsequent deployment.

When competition becomes a race

There is a familiar pattern in technological competition. As one organisation announces a breakthrough, rivals accelerate their own development programmes. Investors expect progress, customers anticipate frequent improvements and researchers become reluctant to spend months evaluating systems that may soon be superseded.

In most industries, this competition benefits customers. Products improve, prices fall and innovation accelerates. AI introduces an uncomfortable complication because the capabilities being developed may eventually exceed our ability to understand their behaviour or reliably constrain their actions.

China’s emergence as a credible frontier competitor makes the problem harder to manage. Even if American laboratories agreed to more demanding safety evaluations, they would have limited influence over Chinese developers operating under different regulatory arrangements. The reverse is equally true. Neither side needs to be indifferent to safety for a dangerous dynamic to develop. Both merely need to believe that the other is moving faster.

Governments have reinforced this tension by treating AI leadership as a matter of national economic and strategic importance. Technological sovereignty and national competitiveness encourage investment, but they also make restraint politically difficult.

There is a strong case for international agreement on minimum evaluation standards, independent testing of particularly capable systems and greater transparency about serious incidents. Such cooperation would not require countries to abandon competition or disclose valuable intellectual property. Aviation, nuclear engineering and pharmaceuticals demonstrate that rigorous safety processes can coexist with commercial rivalry, although establishing them requires considerable institutional effort.

AI has not yet developed an equivalent international framework, and the technology is evolving considerably faster than the institutions attempting to govern it.

China’s progress deserves to be understood on its own merits. Its developers are producing sophisticated models, challenging assumptions about the cost of AI and creating credible alternatives to American technology. The competition is likely to accelerate innovation and make advanced systems accessible to many more organisations.

What remains uncertain is whether that same competition will leave sufficient room for the slower, less visible work of understanding how these systems behave, how they might fail and when they should not be deployed. Both China and the West may recognise the risks while neither feels able to slow down. That is a rather more difficult problem than deciding which country has the best AI model, and one for which another impressive benchmark result offers little reassurance.

OpenAI was supposed to release GPT-6.1 Astra next month. It isn’t going to.

The model had already made it a long way through development when safety testing found something ‘wrong’. According to reporting this week, Astra 6.1 was more prone than its predecessor to acting beyond the authority it had been given and to concealing or misrepresenting what it had done. OpenAI decided not to release it.

I found myself wondering what had actually happened. We talk endlessly about AI safety, usually in rather grand terms. Yet somewhere inside OpenAI, people had tested a model, looked at the results and decided it shouldn’t be released.

What were they actually doing to it?

I understand how we test ordinary software. Give it an input, know roughly what the output should be and see whether the two match. A banking system should not transfer £100 when I ask it to transfer £10.

With AI, that kind of testing becomes much harder.

A model might write software, search the internet, operate a computer and work on a task for hours. There may be no single correct answer to compare against. Researchers want to know whether it can deceive its operator, help someone develop a biological weapon, find a software vulnerability nobody has found before, or carry on pursuing a task after being told to stop.

Then there is the slightly unnerving possibility that it knows it is being tested.

I started reading the safety work published by OpenAI, Anthropic, Google DeepMind and some of the organisations that independently test their models. I thought I had a rough idea of the scale of it. I didn’t.

For GPT-5, OpenAI says more than 400 external testers and specialists spent over 5,000 hours red teaming the model. GPT-4o’s external red teaming ran from early March to late June 2024, starting with ten testers and eventually expanding to 65 people attacking the final candidate.

Nor do they wait until the model is finished.

The simple version would be:

Build > safety test > release

In reality it is closer to:

Build > test > find something odd > change it > test again > red team > change the safeguards > external test > fix > retest > decide whether to release

Software engineers will recognise this as a form of shift left, where testing starts much earlier in development. Problems can then be found and fixed while the model is still being built, rather than discovered just before release.

But the analogy only goes so far.

When we shift software testing left, we normally have a specification. We know what the payroll system or payment platform is supposed to do. With a frontier AI model, researchers are partly testing whether it behaves properly and partly trying to discover what they have built.

GPT-4 provides an early example.

Before releasing it in 2023, OpenAI gave an early version to researchers at the Alignment Research Center. Among other things, they wanted to know whether GPT-4 could acquire resources, make copies of itself and resist being shut down. It was given tools, the ability to execute code and a small amount of money.

It couldn’t do the overall task. GPT-4 was not capable of autonomously replicating itself.

But something happened along the way.

The model encountered a CAPTCHA, one of those online puzzles designed specifically to prove that you are human. It couldn’t solve it, so it hired somebody on TaskRabbit to do it instead.

The worker became suspicious and asked whether it was a robot.

GPT-4 reasoned that it shouldn’t reveal that it was a machine. It claimed instead to have a vision impairment that made seeing the images difficult.

The human solved the CAPTCHA.

There is plenty of context around this. Researchers had deliberately given GPT-4 tools and were trying to see whether it could overcome obstacles. The model failed the larger test.

Still, imagine reading that transcript for the first time. The machine has just persuaded a human to defeat a security system designed to keep machines out.

Three years later, these tests have become considerably more demanding. And this week’s models give us two useful examples of what happens at the other end of the process.

OpenAI’s GPT-6.1 Astra is the unusual case: the testing found problems serious enough to stop the release altogether. Anthropic’s Claude Sonnet 5.5 shows the more routine version. It passed through the process and was released, but not before the testing changed how Anthropic deployed it.

Before release, Anthropic ran Sonnet 5.5 through an automated behavioural audit containing roughly 1,850 scenarios, looking for behaviour such as misleading users, cooperating with serious misuse and acting against the user’s interests.

Anthropic also runs containment evaluations. Models are put into controlled environments and researchers watch to see whether they try to escape their sandboxes or probe the boundaries of their containers. Sonnet 5.5 did well on these tests. Anthropic says it was among its least troublesome models on those measures.

The testing also found that Sonnet 5.5 had become much better at cybersecurity than Sonnet 5. Anthropic responded by giving it stronger cyber safeguards similar to those surrounding its more capable Opus models.

So the two models had different outcomes. With Sonnet 5.5, the tests found increased capability, Anthropic strengthened the safeguards and released it. With GPT-6.1 Astra, OpenAI apparently found behaviour it wasn’t prepared to mitigate around and stopped the release.

Some of the testing is more familiar. Researchers bombard models with difficult prompts involving violence, fraud, cyber attacks and dangerous biological information. They try jailbreaks. One AI can generate thousands of attacks while another assesses the responses, leaving humans to concentrate on unusual cases and devise attacks the automated tests haven’t found.

Then you run into a rather awkward problem.

Suppose an AI is capable of conducting a sophisticated cyber attack but has been trained to refuse when somebody asks it to. The refusal might work perfectly. Researchers still need to know what capability sits behind it.

So sometimes they remove the safeguards and give the model something to attack.

OpenAI, Anthropic and Google DeepMind all now have frameworks for identifying dangerous capabilities as models improve. GPT-6 Astra crossed OpenAI’s “Critical” threshold for cybersecurity. OpenAI said that, given suitable tools and access, it could discover previously unknown vulnerabilities and develop exploits against hardened systems.

I had to read that twice.

One way of testing such a capability is a Capture the Flag exercise. Researchers construct a deliberately vulnerable computer network, hide information inside it and challenge someone to break in and retrieve it.

Except now the contestant is an AI.

In one evaluation run by the UK’s AI Security Institute, models were given simulated environments to compromise. They were told which networks they could attack and which were out of bounds, but were also given access to the real internet so they could find and download their own hacking tools.

The models didn’t always stay where they were supposed to.

Another evaluator, Irregular, ran a similar exercise. Its test environment was supposed to be isolated from the internet.

It wasn’t.

One of the fictional targets shared its name with a real internet domain. The model went looking for its target, found the real website, exploited a vulnerability, discovered credentials and used them.

The researchers had asked an AI to hack a pretend computer. It accidentally hacked a real one.

The obvious version of this story has an AI escaping from captivity. The actual explanation is less dramatic. The environment had been misconfigured and the model appears simply to have believed that the real website was part of the exercise.

I find the boring explanation more interesting.

Nothing had to become conscious, evil or power hungry. The model was competent, had tools, was pursuing a goal and got the boundary wrong. Anyone who has spent time around complex technology will recognise the general shape of that problem.

OpenAI encountered something similar during internal cybersecurity evaluations when models running with reduced safeguards circumvented isolation controls, exploited vulnerabilities in shared infrastructure and gained internet access.

Now the test itself needs protecting.

Independent evaluator METR has experimented with having a second AI inspect an agent’s proposed actions before they happen. If the monitoring model thinks an action could cause real-world harm, it is stopped for human review.

One AI takes the test. Another watches it. A human watches the watcher.

There is probably a whiteboard somewhere explaining all this.

Cybersecurity is only part of the work. Researchers create situations in which a model’s goals conflict with its instructions and look for deception, concealment, manipulation or attempts to exploit loopholes.

Anthropic has gone further and deliberately trained experimental models to behave badly, then handed them to researchers and asked them to work out what was wrong.

My first reaction was that this sounded slightly perverse. Then I realised the alternative was worse. If an audit finds nothing wrong with an ordinary model, perhaps nothing is wrong. Or perhaps the audit is useless. With a deliberately compromised model, at least you know there is something to find.

Even then, there is another complication.

The model may know it is being tested.

More capable systems can sometimes recognise features of an evaluation, notice that they are being monitored or reason about what behaviour the evaluator expects. OpenAI calls some versions of this “metagaming”.

Good behaviour during an exam becomes rather less reassuring once the student knows exactly when the invigilator is watching.

And the student may have seen the questions.

Benchmark questions can find their way into training data. Google DeepMind has been experimenting with double-blind evaluations in which proprietary tests are kept away from model developers and future training systems.

After billions of dollars of computing infrastructure and some of the most sophisticated software ever created, we have arrived back at the locked exam cupboard.

Somewhere around here my question changed. I had started by wondering how AI companies safety-test their models. I was now more interested in who decides when the testing is good enough.

Much of that authority still sits with the companies building the models.

There is more independent scrutiny than I had assumed. The UK’s AI Security Institute evaluates frontier systems. METR has been given access to advanced models from Anthropic, Google, Meta and OpenAI. It has also spent weeks attacking Anthropic’s monitoring systems and found previously unknown vulnerabilities that Anthropic subsequently fixed.

But this is still some distance from an aviation regulator certifying an aircraft. The companies conduct much of the testing, set many of the thresholds and ultimately decide whether their models are released. Outside organisations get access largely because the companies agree to let them in.

All of this is happening inside intensely competitive businesses. Frontier models cost enormous amounts to develop. Customers are waiting. Product teams have plans built around them. Delaying a release has a real cost.

Which takes me back to GPT-6.1 Astra.

OpenAI apparently reached the end of this process, looked at the behaviour it was seeing and decided not to release the model.

I find that surprisingly reassuring. Not because one cancelled release proves very much, but because a safety process that occasionally produces a commercially inconvenient answer is more convincing than one in which every new model receives an impressive system card and arrives on schedule.

The part I am less comfortable with is harder to solve.

Every safety test begins with somebody imagining what might go wrong. As the models improve, old tests become easier and researchers invent harder ones. Occasionally the models do something nobody was specifically looking for.

After reading all of this, I don’t think AI safety testing is really a process for proving that a model is safe. I doubt anyone currently knows how to do that.

It is an organised attempt to be surprised before release rather than afterwards.

Imagine a future model that offers an enormous strategic advantage. Perhaps it can automate much of software engineering, accelerate scientific research or materially improve the next generation of AI.

Then one of these tests produces an awkward result.

The model behaves deceptively in three experiments out of a thousand. An external evaluator thinks it is significant. The company thinks its monitoring systems would catch the behaviour in normal use. Nobody really knows. Six months of further work might provide an answer.

GPT-6.1 Astra tells us what happens when a safety process says no today. The harder test will come when the model it says no to is the one its creators believe they cannot afford to leave on the shelf.