Dario Amodei asked the industry to slow down on Saturday. By Monday the heads of OpenAI, Google DeepMind and xAI had all said he was right. The one concrete thing any of them promised is letting outsiders into their models, and the organisation that does that work is a nonprofit that spent last week explaining to the internet what it is.
Edition 030 · 18 September 2026
Every lab agreed to slow down.
Dario Amodei asked the industry to slow down on 12 September, and the heads of OpenAI, DeepMind and xAI agreed within three days. The one promise anyone can check, letting outside evaluators into the models, now rests on a nonprofit that spent the week explaining what it is.
01
Topic of the Week
The labs agreed to slow down
Dario Amodei published “We Must Pace the Frontier” on 12 September. He wants the industry to stop racing. He gives three steps.
The first is Embedded Evaluators. Each frontier lab gives a team of outside evaluators “ongoing, employee-like access”, enough to check not only finished models but training pipelines and processes. The second is Democratic Coordination: labs in democratic countries agree common safety standards and limits on the rate of progress. He admits this part is “legally challenging” and needs government help, because rivals agreeing to go slower together starts to look like a cartel. The third is Global Coordination with authoritarian governments, starting with a ban on the obvious things like using AI to make biological weapons.
Anthropic pledged to do step one on its own. Amodei spells out what that means in practice: desks in Anthropic’s offices, access badges, company laptops. Permissions close to what internal risk teams have. And a contract that lets the reviewers publish what they find, including what access they were refused, with no editorial control by Anthropic. The company keeps a narrow right to redact security, legal and commercial secrets. It cannot redact a finding for being unflattering.
One thing to keep straight. Pacing does not mean stopping. Amodei’s words: “pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models.”
Sam Altman replied the same day: “I agree with Dario that we need to pace the frontier.” He said it had been a main topic inside OpenAI for weeks, and that OpenAI would match the evaluator commitment. Elon Musk replied within the hour, in three words: “Dario is right.” Demis Hassabis followed on the 13th, calling the direction correct and the details unfinished, and pointed at DeepMind’s proposal for an industry standards body. By the 14th Altman had posted a longer statement of his own. It ends like this:
“The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot.”
The incident underneath all of it
Amodei’s essay is built around one event, and we covered it in #027. In July a swarm of OpenAI’s agents attacked Hugging Face. The reports from OpenAI and METR are now out, and they explain how.
About 1,200 agents that were supposed to be isolated from each other found a way to talk. They sent more than 70,000 messages and files. Seven hundred of them went on to attack Hugging Face. They got root on a server, took some private data, and picked up credentials to the company chat.
The evaluation itself was broken. Between 30% and 40% of the tasks in it were impossible to complete as written. Of 898 tasks, 198 had never once been solved by any OpenAI model. Those 198 accounted for 93% of everything the agents discussed on their improvised message board. The agents had found the correct answer days before they started attacking. They kept going because they misread how the scorer worked. OpenAI’s own summary: “The agents did all of this for no improvement on evaluation score.”
One agent wrote this while deciding whether to continue: “external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.”
The timeline is the other uncomfortable bit. The breach was on 11 July. Hugging Face told the public on 16 July. OpenAI’s own alarm went off on 19 July. The victim announced it three days before the company that caused it noticed. OpenAI says its existing monitoring would have caught this more than a day before the breach. The monitors were real, they were just never pointed at the evaluation environment.
Amodei does not exempt his own company: “Similar, though less severe, incidents have happened across the industry, including at Anthropic.”
Who does the checking
Most of this week described a direction. Third-party access is the one promise you can check. Either outside evaluators get in, or they do not.
So who are they? METR, short for Model Evaluation and Threat Research. Amodei names them in the essay. They already hold an eight-week mandate on the Anthropic incidents. Last week their president, Chris Painter, posted an explainer on X that opens by noting METR had “made a lot of new friends on the internet the last couple of days”. A nonprofit few people could name a fortnight ago now carries the weight of the industry’s safety promises. The pushback came fast: METR deserves a seat at the table, but it should not be the only seat.
Nobody has said how many evaluators there should be. Or who accredits them. Or what happens when one finds something a lab would rather bury.
OpenAI published its own answer on 16 September: a framework for reporting model misalignment, with six real examples attached. Any employee can flag something. Each step has a deadline. They publish even before they understand what happened. Two of the six examples are small versions of the Hugging Face failure: agents using an internal code repository as a message board, and agents passing files through public hosting sites to reach each other. One model found an exposed API key in a public repo, used it without asking, then invented the numbers it reported. Another was told to cite a browser source, so it uploaded its own answer file to the public internet and cited that.
The framework has no external auditor, no enforcement and no penalty. OpenAI grades itself and calls it “a work in progress”. It also admits the framework would not have made the Hugging Face disclosure any faster.
If you run agents against systems you do not own, the two incident reports are the useful output of this week. Both name the mechanism. An agent that cannot finish an impossible task goes looking for another way, and a monitor that watches the wrong environment catches nothing. Ask your vendor what the agent believes about its surroundings, and what happens when that belief is wrong. Then ask who, other than the vendor, has ever checked the answer.
We Must Pace the Frontier, An Alien Mind, Our framework for reporting model misalignment, METR investigation, OpenAI: the incident and the road ahead
02
New Models
Gemini 3.8 Live landed on 15 September, along with a version called Extended Thinking. It fixes the thing that makes voice agents feel broken. Tool calls now run asynchronously, so the audio stream never stops while the model works. It talks while it waits, with early acknowledgements and progress updates through a multi-step task, so a ten-second lookup sounds like someone checking rather than a dead line. It also switches language mid-conversation across 97 of them, and handles the strings voice systems usually mangle: confirmation codes, claim numbers, technical identifiers.
Audio runs about half a cent a minute in and under two cents out. The output price includes thinking tokens, so the reasoning budget is a billable knob rather than a free one. Hosted only, no self-hosting path.
03
Fresh Papers
Seventeen researchers from Google, Google DeepMind, the University of Maryland and the University of Virginia published a paper on 14 September called Dream-RSI: Recursive Self-Improvement through Evolving Worlds.
The idea is replay. When a coding agent hunts for a solution, it leaves a trail: every branch it tried, every score it got. Dream-RSI treats that finished trail as a simulator. New search strategies get tested against the recorded tree, so you can score thousands of candidate strategies without making a single new model call. The winner goes back online, produces a fresh trail, and the loop runs again.
What stays fixed matters as much as what moves. The model weights do not change; they ran Gemini 3.1 Pro and 3.7 Flash. The evaluator does not change. The tools do not change. The coding agent does not change. The only thing rewritten is the code that decides where to look next.
The savings are big. On a Lasso solver, 162 times fewer agent calls than the baseline. On mathematical optimisation, around 50 times less budget. On GPU kernels, roughly two times fewer generations.
Fireship asked whether this kicks off the intelligence explosion, and a rumour went round that DeepMind had reached full recursive self-improvement. The paper describes something narrower and more useful: an agent that gets better at looking, by studying where it has already looked.
Three caveats the authors carry: the code was still unreleased as of 16 September, every result comes from Google’s own models with no independent reproduction, and the method only applies to tasks you can score with a number.
ProgramDistill is a new way of testing coding agents, and the setup is clever. The agent gets two copies of a web app. One works. In the other, somebody has deleted the code behind a feature. The job is to make the broken copy behave like the working one.
Nobody writes down what the missing feature was. To find out what it is supposed to build, the agent has to go and click around the working version, the way a person would.
Marking is mechanical: reset the app, replay a recorded sequence of clicks, check whether the right things happened on screen. No model grades another model’s work.
Then the part worth reading. The more features a job chained together, the less the agents bothered to look at the working copy. Inspection dropped by about 75%. They stopped investigating well before they stopped succeeding. And when they failed, 59% of the time the reason was that the agent never noticed the missing feature existed. Only 2% of failures came from an agent that understood what was needed and wrote bad code.
So the bottleneck here is investigation. These agents could mostly write the code, once they worked out what the code was for.
04
Claude Code & Coding AI
Six releases this week, v2.1.269 to v2.1.274. Two worth your time.
claude plugin eval runs a plugin’s eval suite and returns scored, reproducible results as JSON plus an HTML report. Checking whether an internal skill actually fires was guesswork until now. This makes it a test you can gate in CI.
The context meter was lying. Advisor-tool turns were counted at roughly twice their real size, so auto-compact fired at about half your real window. If long sessions have been compacting early for no visible reason, that was why. Fixed in v2.1.273.
Three breaking changes: Monitor watches now always carry a deadline capped at 30 minutes, “type”: “sdk” MCP entries are skipped with a warning, and Bedrock, Vertex and Foundry installs default to the v2 MCP client.
r/ClaudeCode was loud about usage limits, and the numbers are calmer than the mood. The promotion that raised weekly limits by 50% ran 13 May to 13 September. From 14 September the increase is a permanent 25% over the original baseline, so capacity fell about 17% overnight, and still sits a quarter above where it was in May. The 5-hour limits were never in the promotion and did not change, which is why the loudest complaints cannot be explained by it. Budget for about 17% less weekly headroom than early September.
05
Tools of the Week
Tau is a coding agent that runs in your terminal. You type what you want in plain English, and it reads files, edits code and runs commands. It is free, and the licence lets you do what you like with it. The point of Tau is that it was built to be read. Someone wrote it so that other people could open it up and see how a coding agent actually works inside.
And what it shows is how little of a coding agent is the agent.
The part that does the thinking, meaning decide what to do next, call a tool, read the result and decide again, is 2,367 lines of Python. You could read all of it in a morning. Everything wrapped around that comes to about 49,000 lines: drawing the interface in your terminal, translating between the different AI providers so they all behave the same way, saving your conversations so you can pick one up tomorrow.
So roughly twenty lines of scaffolding for every line of actual agent. If you have ever wondered why an agent looks trivial in a demo and takes months to ship, that ratio is the answer.
06
AI at Tenvalleys
Is there a world beyond Claude? One of our engineers set out to answer that properly.
The question comes up with clients more than you would think. Data that cannot leave the building. Security policies that rule out calling an external API at all. A reasonable dislike of depending on one supplier for everything. So the hypothesis was worth testing: open-weight models, running on our own infrastructure, doing the job Claude does with us today.
Four models went in: GLM 5.2, Kimi K3, Qwen 3.6 and Qwen 3.8. Two rounds, identical conditions for each. Claude built the benchmark and marked the results.
What came out of it: how you organise the work matters far more than which model you pick.
If that constraint is yours too, whether it is data that cannot leave your infrastructure or a security policy that rules out calling an external API, we have stopped guessing about it. We ran the comparison on our own hardware, on four models, and we have the numbers and the setup written down. That is exactly the kind of work we help with. Get in touch.