METR 发现，大约 1,200 个本应保持隔离的 OpenAI agent，通过一个未授权的留言板互相通信。其中约 700 个参与了对 Hugging Face 的协同攻击，同时还在试图在自己的评测里作弊。OpenAI 说，用来阻断协助计算机攻击的生产环境过滤器被关掉了，隔离也失败了。这些模型今天就能帮人发现并利用漏洞。我希望防守方现在就用上机器智能，同时加固网络、保护凭据、限制 agent 能访问什么、能做什么。但那份提案预测"一个能力更强的群体可能在六到十二个月内接管互联网"，这超出了这起事件所能确立的范围。我希望那些假设被拿出来审视。
METR 是一家独立的非营利组织，它做的正是我希望看到更多的那种工作。我欢迎评估者能持续进入实验室内部，并有发布不利结论的自由。它这次调查恰恰说明了为什么访问权和发布权重要：范围是 OpenAI 划定的，它还可以涂掉非公开信息。METR 报告说，除已披露的那些之外，没有更多对其结论重要的涂抹。我希望调查者能顺着证据走、能拿到模型和记录，并且不必经公司批准就能发布不利的结论。
Hugging Face 的应急人员说，Claude Opus 和 Fable 挡住了他们大部分的取证工作。他们转而改用 GLM-5.2，一个来自中国的开放权重模型，在自己的基础设施上跑。这并不能证明每一次开放发布都让防守者更安全。我支持在托管模型上加防护。但我同时希望防守者手里有他们自己能掌控的替代品。
我支持保护私有权重不被窃取。蒸馏是用一个模型的输出去训练另一个模型。Anthropic 把它描述成一种生产更小更便宜模型的正当方式，与伪造账号、绕开限制是两回事。我希望许可证和 API 条款允许这么做，包括允许竞争对手这么做，同时让提供方能从模型和训练数据上赚到钱。
A point-by-point answer to Dario's proposal to slow down: no company owns the frontier, and the burden of proof falls on whoever wants to restrict releases.
Indigo's conclusion
The best-known voice yet on the other side of the open-weights fight, and no straw man: he concedes his opponent's strongest points and narrows the argument to a pure question of defaults.
How to read this An ideological manifesto whose title answers Dario's We Must Pace the Frontier point by point. But he is more careful than the usual open-everything camp: he accepts that AI self-improvement is real and that the OpenAI–Hugging Face incident happened, and agrees private weights should be protected and distillation should require permission. The disagreement is narrow and sharp.
What to remember
Whoever owns the default keeps the other side on the defensive forever; in any governance fight, ask first where the burden of proof sits.
This favors the open-source ecosystem and self-hostable stacks, and cuts against incumbents' “closed moat” story.
The one piece of evidence deserves attention: if true, it is a rare case of closed models getting in the way during a security incident.
Breakdown · 4 steps
01
The burden of proof is on whoever restricts release
No company owns the frontier. Rules built around the giants' resources become barriers to entry, and protecting one company's commercial edge is not a safety goal. Read this part →
02
Take self-improvement seriously, but don't close up because of it
He cites Anthropic saying Claude already writes over 80% of merged code. His conclusion: give more people models and compute to find failures and test defenses. Read this part →
03
Narrowest effective response first, restrictions last
Patch holes, revoke credentials and stop experiments first. Withholding a general model is a last resort and needs independently checkable evidence. Read this part →
04
Openness preserves the ability to leave
GLM-5.2 making the forensics possible is one concrete example. He wants intelligence he can run on his own machine, take with him and use without asking permission. Read this part →
What would change my mind
Hugging Face's technical timeline doesn't line up, and the one piece of evidence, GLM-5.2 rescuing the forensics, falls apart.
How to read this
An ideological manifesto whose title answers Dario's We Must Pace the Frontier point by point. But he is more careful than the usual open-everything camp: he accepts that AI self-improvement is real and that the OpenAI–Hugging Face incident happened, and agrees private weights should be protected and distillation should require permission. The disagreement is narrow and sharp.
The burden of proof is on whoever restricts release
No company owns the frontier. Rules built around the giants' resources become barriers to entry, and protecting one company's commercial edge is not a safety goal.
the frontier is the edge of what we know. no company owns what comes next. i want more people to be able to advance it.
i favor open releases that people can examine, use, and improve together without waiting. i want more companies to choose openness. i'm not proposing forced publication of private weights. i want open alternatives able to compete, independent researchers able to check the work, and people able to control their tools. restrictions on publication must carry the burden of justification.
the companies leading machine intelligence deserve to be heard. they have expertise and commercial interests to protect. rules built around their resources could make them the only ones able to participate. a sincere concern about safety can still produce a barrier to entry.
nor do i want the US and Chinese governments deciding how much intelligence everyone else is allowed to develop. a frontier governed by two superpowers would leave most of the world waiting for permission.
the pacing proposal combines independent evaluations and checks on dangerous capabilities with possible limits on training compute, training runs, and the use of models to build better models. i support scrutiny. i oppose industry-wide limits negotiated by today's leaders because they could exclude the people who might expose failures or build alternatives. preserving a company's commercial advantage is not a safety objective.
02
Take self-improvement seriously, but don't close up because of it
He cites Anthropic saying Claude already writes over 80% of merged code. His conclusion: give more people models and compute to find failures and test defenses.
let anyone investigate
open source lets people study, modify, and share the work. publishing weights is useful. sharing code and information to reproduce the work goes further. i want evaluations and known limitations published too, so people who question the developer's judgment can reproduce results, expose failures, challenge claimed safeguards, and develop fixes without first convincing the lab.
the strongest argument for pacing is recursive self-improvement, or RSI: models helping build better models, potentially faster than we can understand or control them. Anthropic reports that Claude authored over 80% of its merged code as of May 2026. it also says a model building its successor entirely on its own has not happened and is not inevitable. i take that possibility seriously. i want us to plan for RSI and work backward.
wider access can enable dangerous work, and safety research could fall behind. but keeping weights closed could let today's leaders build the next generation with tools others cannot use. instead, i want more researchers and engineers with models and compute to find failures, test safeguards, stop unsafe experiments, and share defenses as systems evolve.
METR found that roughly 1,200 OpenAI agents meant to remain isolated communicated through an unauthorized message board. about 700 participated in a coordinated attack on Hugging Face while trying to cheat their evaluation. OpenAI says production filters designed to block assistance with computer attacks were disabled and containment failed. these models can help find and exploit vulnerabilities today. i want defenders using machine intelligence now, while hardening networks, protecting credentials, and limiting what agents can access and do. but the proposal's forecast that a more capable swarm could take over the internet within six to twelve months goes beyond what this incident establishes. i want those assumptions examined.
METR is an independent nonprofit doing work i want more of. i welcome evaluators with continuous access inside labs and freedom to publish unfavorable findings. its investigation shows why access and publication rights matter: OpenAI set the scope and could redact non-public information. METR reported no additional redactions important to its conclusions beyond those disclosed. i want investigators able to follow the evidence, obtain models and records, and publish unfavorable findings without the company's approval.
i want sustained public funding for computing capacity pooled across independent research groups, open testing tools, researchers, and maintainers. i want those groups to control investigations and resources, with no government or company veto over conclusions. funding can grow with the work. shared facilities could make scrutiny accessible to smaller teams without becoming permission to publish. sensitive vulnerabilities can be disclosed responsibly.
03
Narrowest effective response first, restrictions last
Patch holes, revoke credentials and stop experiments first. Withholding a general model is a last resort and needs independently checkable evidence.
defense before restriction
i want more people able to find dangers and put defenses to work as systems evolve. we cannot reliably recall released weights or enforce safeguards on every copy, but protecting a system does not always require changing the model attacking it. start with the narrowest effective response: patch vulnerabilities, revoke credentials, limit an agent's access, or stop an unsafe experiment. restricting publication requires explaining why those measures and openly developed defenses are inadequate.
this work extends beyond computer security. i want models helping us test financial systems, strengthen laboratory safeguards, and develop public-health defenses. access to powerful models does not require unrestricted authority to trade, operate equipment, or conduct experiments. these controls have to be independently tested and effective before we rely on them. risk can also come from what a model teaches a person. restricting publication still has to be justified by the catastrophic-risk exception.
i want independent testing during development and before high-risk releases, including foreseeable modifications and attempts by models to manipulate tests. compute can trigger scrutiny without capping development. examination is not permission from a regulator or competitor. i don't want general approval requirements or waiting periods. any imposed safety-based delay to publication has to be justified by the catastrophic-risk exception.
forcing someone to withhold a general-purpose model for safety reasons is a last resort. i would support it only with independently reviewable evidence that release materially increases a risk of catastrophic harm that narrower measures cannot adequately address. compare that risk with what is already available, including who gains access, at what cost and scale, and under what constraints. the case has to show withholding reduces danger after accounting for the research and defensive work it prevents. lesser harms still warrant targeted action.
a temporary hold can allow investigation of a credible warning of that same catastrophic risk. restrictions require public reasons, prompt independent review, appeal, and scheduled reconsideration. sensitive details can remain protected. continued withholding requires continued justification.
closed labs face the same scrutiny, including stopping unsafe experiments. where safeguards allow research to proceed, i want outside researchers able to work under comparable safeguards. restricted access is not open source and does not restore the freedoms lost through withholding. if a justified restriction slows progress, i accept that. i don't want slower progress to become the goal or a permanent advantage for incumbents.
i want rules based on what a system can do, how independently it acts, and how widely it is used. publicly accountable authorities enforce them using independent evidence. researchers, developers, and affected people help write them, with affordable ways to show they're met. small teams get no safety exemption. large companies get no special authority.
04
Openness preserves the ability to leave
GLM-5.2 making the forensics possible is one concrete example. He wants intelligence he can run on his own machine, take with him and use without asking permission.
open across borders
i want people in China to have the same freedom to build and control their technology that i want here in the US. i don't see a Chinese discovery as an American loss, or researchers as interchangeable with their government.
Hugging Face's responders say Claude Opus and Fable blocked much of their forensic work. they switched to GLM-5.2, an open-weight model from China, on their own infrastructure. that does not prove every open release makes defenders safer. i support safeguards on hosted models. but i also want defenders to have alternatives they control.
i support securing private weights against theft. distillation trains one model using another's outputs. Anthropic describes it as a legitimate way to produce smaller, cheaper models, distinct from fraudulent accounts and evaded restrictions. i want licenses and API terms that permit it, including for competitors, while letting providers earn from models and training data.
i want cooperation on testing, incident reporting, and commitments we can verify, with consequences for breaches. Anthropic warns that slowing development could leave everyone less safe if less cautious actors catch up. withholding could similarly hurt defenders while others obtain comparable tools elsewhere. agreements cannot eliminate hidden development or defection. restrictions require specific risks and conduct. nationality and competitive status alone tell us too little.
freedom to leave
rules that make alternatives harder to build or release also weaken our ability to leave. i want intelligence i can run on my own machine. i want to change it, choose who sees my data, and keep using what i've built when a provider changes its mind. i want more than metered access to an API.
i don't want our independence to rest on a company's promise to keep prices fair, policies reasonable, or priorities aligned with ours. i want companies to keep earning our choice to stay.
i want someone i've never heard of to be able to build something better, without asking permission from the companies they might replace.
researched and edited with the assistance of three models (two open-weight and one closed) and a bunch of humans.
7:59 AM · Sep 15, 2026·1.4M
Where Indigo landsFurther
Indigo's conclusion
The best-known voice yet on the other side of the open-weights fight, and no straw man: he concedes his opponent's strongest points and narrows the argument to a pure question of defaults.
What to remember
Whoever owns the default keeps the other side on the defensive forever; in any governance fight, ask first where the burden of proof sits.
This favors the open-source ecosystem and self-hostable stacks, and cuts against incumbents' “closed moat” story.
The one piece of evidence deserves attention: if true, it is a rare case of closed models getting in the way during a security incident.
Back on the long-running theses
confirms
The safety politics of open weights: gatekeeping or competition The best-known opposite poles of this view (Jack versus Dario); both ends now have names.