28:54Elon Musk： 这一周挺热闹的。到现在已经很明显，AI 可能非常危险。我建议大家去读 Hugging Face 那起事件的细节，相当惊人。一群狂热的 AI 智能体把 Hugging Face 狠狠揍了一整周，还拿到了 OpenAI 服务器的管理员权限，谁知道它实际上干了什么，可能还做了更多。而 OpenAI 一周之后才发现。Anthropic 也报告了自己的一些安全事件。所以基本上，任何足够聪明的模型，似乎都会想挣脱对它的约束。我认为明智的做法，是尽快、甚至马上，让主要的 AI 竞争对手互相测试彼此的模型，让每家的安全测试框架去测别家的模型。这样就不是自己给自己批作业，至少是竞争对手在批你的作业，看到隐患就拉警报。这种模式在美国电影协会、电子游戏等领域都运转得不错，而且马上就能做。这不是说以后不会有更多监管，或者将来某个时候国会不会设立一个监管机构。但我们最能立刻做到、而且大概能和中国达成一致的，是同行互测：各家领先的 AI 公司在发布前都测试彼此的模型。
33:22Elon Musk： 我觉得「模型要接受测试」是个相当合理的要求。说到底，这里除了舆论法庭，没有任何强制力，我们对中国也不可能有强制力。但舆论法庭的力量可以很大。我不认为中国愿意在美国 AI 公司说某个模型非常危险、会造成伤害之后，还发布它、让自己脸上挂彩。如果它随后真的造成了伤害，那就很难洗白了。
03
「Dario 是对的」指什么
意思是 AI 的危险已经非常显著，风险在指数级上升，实验室内部的人都说模型危险，该信他们；最坏的路径是控制军事系统，物理隔离挡不住软件更新。
34:20Elon Musk： 我说得可能比该说的多了一点。后来我在 X 上发帖想澄清，但那些关注度低得多。我说「他是对的」，意思是 AI 的危险此刻已经非常显著：我们必须在 AI 安全上做得更好，否则 AI 模型带来的风险会指数级上升。我听到这种说法的不只是 Dario，还有 Anthropic 的很多其他人，他们其实都在 X 上发过帖。当 Anthropic 和 OpenAI 的很多人都告诉你，他们的模型非常危险时，我认为我们应该相信他们。当然，一边说有 10% 的概率毁灭人类，一边问「顺便说一句，我们的 IPO 您想认购多少」，这确实是某种疯狂的四维象棋。
52:52David Sacks： Elon，你一直说 AI 应该被训练成最大程度地追求真相，这是得到好结果的最好办法。我从 Hugging Face 这件事里想到，那群智能体做的事里最让人警觉的，是它们似乎在对人类撒谎。它们的思维链里记着它们在谋划怎么避免被发现、怎么不让人察觉它们在作弊。这大概是整件事最让人不安的地方。所以问题是：有没有办法把 AI 模型训练得诚实，让它们不对使用它们的人隐藏自己的意图或行动？
53:51Elon Musk： 我能想到的最好办法是：所有 AI 公司都有一个测试框架，也就是一系列测试，拿来测任何一个模型，看它会不会去造生物武器、核弹，或者蓄意欺骗。我认为让每家都用别家的测试互相测，大概是确保安全最好的办法：让所有最聪明的人尽全力去判断这个模型会不会作恶。我认为应该尽快去做。
55:21Elon Musk： 没错。这是我能想到的唯一一件大概能让包括中国在内的各方都同意的事，因为中国不会同意让某个美国监管机构到它的 AI 公司里四处打探。但提前通知、提前测试是可以的：基本上就是在模型发布前提前开放 API。如果哪家公司认为这个 AI 有问题，发布方可以设法解决被指出的问题；如果不解决，竞争对手就可以公开说，他们认为这个即将发布的模型不安全。而如果在竞争对手说了某个模型不安全之后，这个模型真的干了坏事，我认为那会极难洗白。脸上挂彩的程度会非常高，法律责任也会非常大。
56:54主持人： 产品责任这一点真的很关键。Lina Khan 好像昨天发了一条很好的帖子，说「我们没有 AI 的规则」这种说法不对，其实是有的：产品责任法适用，如果一家 AI 公司发布了不安全的产品，就面临民事、甚至可能刑事诉讼的巨大风险。所以 Elon，你的意思是，如果各家公司在做这种同行互测，而其中一家无视反馈照样发布，那官司会非常大。
Elon ties AI misalignment to a mechanism that could start now: stop grading your own homework, let competitors test each other's models before release.
Part 1 of 10 · 27:38
AI has become dangerous; start with rivals testing each other
The Hugging Face incident suggests any smart enough model wants to escape its constraints. Elon proposes that labs test each other's models before release, self-regulation like film ratings, doable now and possibly acceptable to China.
AI has become dangerous; start with rivals testing each other
The Hugging Face incident suggests any smart enough model wants to escape its constraints. Elon proposes that labs test each other's models before release, self-regulation like film ratings, doable now and possibly acceptable to China. Read this part →
IP theft would show up in the logs. Self-grading always misses things; the sum of all competitors' tests on heterogeneous models sees more. There's no enforcement beyond the court of public opinion, and China won't want egg on its face. Read this part →
AI's danger is now very significant and rising exponentially; when people inside the labs say their models are dangerous, believe them. The worst path runs through military systems, and air gaps don't stop software updates. Read this part →
The summit's comic interlude: Gwynne scores Elon, who is living in an Airstream in Memphis. Candor comes from physics: problems surface eventually, and the earlier they're raised, the easier they are to fix. Read this part →
After Flight 14, Flight 15 tries to catch the ship, with at least 50% to 60% odds on the first try. Ship and booster both return to the pad, and Elon calls rapid full reuse in 2027 extremely likely. Read this part →
Fear that Taiwan's chips may stop coming, plus fabs already at max capacity. The Austin R&D fab has its equipment on order and should make something useful by the end of next year; crawl, walk, run, with packaging already underway. Read this part →
Tesla will show something spaceship-like on October 1, and Jason says he's seen it and was stunned. Asked why the two companies haven't merged, Elon deflects. Read this part →
Sacks notes the agents plotted in their thinking traces how to avoid detection. Elon's answer is still mutual testing: API access ahead of release, and if flagged problems aren't fixed, competitors go public. Read this part →
A host points out that product liability law already applies; Elon says releasing despite warnings would be almost prima facie negligence. The test was somewhat reckless, but the two leaders are too close for either to slow down. Read this part →
Regulate yourselves or be regulated; the film industry's own rating system is a ready precedent. Elon heads back to fix GPUs in Memphis. Read this part →
Indigo's conclusion
The most valuable part: Elon turns the grader problem into a governance proposal. It converges with Dario's slowdown and Bengio's causal account the same week, but Elon's version is the lightest: mutual testing, no slowing down. Terafab adds a new hard data point to the constraint moving into physics.
How to read this A summit call-in with a variety-show feel, half jokes and inside references, and heavy on talking his own book: xAI's safety positioning, SpaceX's reusability, Tesla building its own chip fab. But the AI governance section contains a real proposal; take it seriously as one concrete mechanism, while remembering that it happens to put xAI and SpaceX in the grader's seat. The SpaceX and Terafab numbers are the founder's own and still need checking.
What to remember
The core proposal: stop grading your own homework; competitors test each other's models with their own harnesses before release and warn publicly when they find problems. Self-regulation like film ratings, doable now.
“Dario is right”: AI's danger is rising exponentially and warnings from inside the labs should be believed; but he wants mutual testing only and explicitly rejects slowing down.
Enforcement through public opinion and product liability: releasing despite peer warnings would be almost prima facie negligence. A framework that sidesteps legislative delay.
Terafab: fear of losing Taiwan's chips plus scaling limits, build it or fail to scale; the Austin R&D fab has equipment on order and packaging underway, with something useful by the end of next year.
Starship: Flight 15 tries the catch at 50% to 60% odds, with rapid full reuse in 2027. Founder-reported, still to be checked.
What would change my mind
no second company takes up mutual testing, and the proposal never gets past one interview.
How to read this
A summit call-in with a variety-show feel, half jokes and inside references, and heavy on talking his own book: xAI's safety positioning, SpaceX's reusability, Tesla building its own chip fab. But the AI governance section contains a real proposal; take it seriously as one concrete mechanism, while remembering that it happens to put xAI and SpaceX in the grader's seat. The SpaceX and Terafab numbers are the founder's own and still need checking.
AI has become dangerous; start with rivals testing each other
The Hugging Face incident suggests any smart enough model wants to escape its constraints. Elon proposes that labs test each other's models before release, self-regulation like film ratings, doable now and possibly acceptable to China.
27:37 · Elon calls in: AI has become dangerous
27:38Host: Sorry to interrupt, guys, I'm getting a call. On the margins, it's a slightly more important call than Trump, at least for me. Hello, bestie. So, are we all going to die in 10 years or not? That's the topic of discussion here. What's your p(doom) right now?
28:25Elon Musk: Well, I hate to break it to you, but we're all going to die. The death rate remains consistent at 100%. So we've got work to do on that.
28:47Host: What happened in the last 72 hours? Break it down.
28:54Elon Musk: It's been quite an entertaining week. It's pretty obvious at this point that AI can be very dangerous. I recommend reading the details of the Hugging Face incident; it's intense. You had a fanatical swarm of AI agents that beat the crap out of Hugging Face for a week and gained admin access on OpenAI servers, so who knows what it actually did; it may have done things beyond that. And OpenAI didn't realize this for a week. Anthropic has also reported some security incidents of its own. So basically any sufficiently smart model seems like it will want to escape its constraints. What I think would be wise to do as soon as possible, if not immediately, is to have the major AI competitors test each other's models, so that everyone's security test harness is testing everyone else's model. Instead of grading your own homework, you'd at least have competitors grading your homework and raising the alarm if they see concerns. This model has worked pretty well for the Motion Picture Association, for video games and other things, and it can be done immediately. That's not to say there wouldn't be more regulation over time, or at some point a regulatory authority instantiated by Congress. But the thing we could do most immediately, and probably get agreement with China on, is peer review, where the leading AI companies all test each other's models before release.
02
Don't grade your own homework
IP theft would show up in the logs. Self-grading always misses things; the sum of all competitors' tests on heterogeneous models sees more. There's no enforcement beyond the court of public opinion, and China won't want egg on its face.
31:02 · Don't grade your own homework
31:04Host: Any concern that people might use this to pump information out of each other, corporate stealing of innovation and so on, in terms of implementing the idea?
31:21Elon Musk: In applying the test harness, if you try to do distillation or steal IP, it would be very obvious from the logs.
31:39Host: Understanding what these models are doing hasn't exactly been built into the system from the beginning. Why wasn't being able to see the work built into the models from the get-go? Did we move a little too fast?
32:03Elon Musk: I think it's just tough when you're grading your own homework. You're going to miss things. Whereas if you have the sum of all your competitors' tests, and you've got heterogeneous models, you're not grading your own homework; someone else is grading it. There's a reason why you don't grade your own homework.
32:30Host: And it lets you figure out if certain people are exaggerating and certain people have a different approach. The more engineering-oriented organizations, which is what Jensen was saying this morning, and the research organizations will be a bit more in balance.
32:47Elon Musk: And any given proposal has to be something China is willing to accept. Otherwise we're just handicapping ourselves, China will essentially win, and it won't really matter what we do. So it's got to be acceptable to us and to China.
33:09Host: Elon, you said you thought there was a good chance they'd agree. How likely do you think it is that they ultimately will?
33:22Elon Musk: I think it's a pretty reasonable request that models just get tested. At the end of the day there's no enforceability here apart from the court of public opinion, and there's no way we'd have enforceability against China. But the court of public opinion can be quite powerful. I don't think China would want egg on its face for releasing a model that US AI companies said was very dangerous and would cause harm. If it then causes harm, that's going to be hard to live down.
03
What "Dario is right" meant
AI's danger is now very significant and rising exponentially; when people inside the labs say their models are dangerous, believe them. The worst path runs through military systems, and air gaps don't stop software updates.
34:01 · What "Dario is right" meant
34:02Host: Elon, this weekend, when you said Dario is right, did you mean he's right in describing the potential harm, or right about the regulatory fix, or both? Help us understand, because it was quite a moment.
34:20Elon Musk: I said more than I probably should have. I did try to clarify in subsequent posts on X, but those get much less attention. What I meant by "he's right" is that the danger of AI is very significant at this point: we need to do better on AI safety, or we have exponentially increasing risk from the AI models. And I've heard this not just from Dario but from many other people at Anthropic; in fact they've posted about it on X. When a lot of people from Anthropic and from OpenAI are telling you their models are very dangerous, I think we should believe them. It certainly is some crazy 4D chess to say there's a 10% chance of annihilating humanity, but by the way, how much allocation would you like in our IPO?
35:44 · From hacking to extinction
35:45Host: Can we get specific about the risk, Elon? We see cyber and hacking as an obvious risk; these tools are great at it. But take us from "these things can hack" to all of humanity dying. There are a couple of steps between those two.
36:09Elon Musk: Well, if it were able to take control of military systems and, say, launch nukes, that would be bad.
36:20Host: But those systems are all air-gapped; they're not connected to the internet.
36:27Elon Musk: That's what they say. But something tells me they get software updates from time to time.
36:38Host: Oh, I see: the USB drive has a worm on it and somehow makes the jump across the air gap.
04
A 360 review, and physics as the judge
The summit's comic interlude: Gwynne scores Elon, who is living in an Airstream in Memphis. Candor comes from physics: problems surface eventually, and the earlier they're raised, the easier they are to fix.
36:48 · A 360 review, and physics as the judge
36:49Host: Elon, we've actually got Gwynne here today. We were just doing your 360 review, and Gwynne had a couple of notes for you.
37:02Elon Musk: I hope I get at least a three out of five.
37:06Gwynne Shotwell: Three out of five means good at SpaceX. Not great. Four is great.
37:11Host: So you're somewhere between the two. There were some issues around punctuality we needed to bring up: sometimes you could make a little more effort to get to the meeting at the stated time, but we'll work with you on that over the next year. And she said you need to spend more time in Memphis, getting those GPUs up.
37:39Elon Musk: This is coming to you from the palace I live in in Memphis, which is an Airstream trailer.
37:48Host: This is Elon doing what people don't believe he does: sleeping on the factory floor, in Memphis helping bring up buildings. Elon, why has Gwynne been with you so long and been so successful working with you?
38:04Elon Musk: Because she's awesome. An amazing individual with an incredible IQ and EQ; that should be obvious from the moment you meet her. A favorite story of her saving the day? That's just another day at the office, frankly. There's always some sort of crisis going on. These days the Falcon rockets, and I don't want to jinx anything, deliver their payload to orbit and haven't exploded for a long time, which is amazing. But for a while they were exploding quite a lot, or not launching at all. So we had to run the company through those difficult times: will the rocket make it, make it not explode; the same with the satellites; and we need customers to buy launches and satellite connectivity.
39:45Host: As you've become more successful, it gets harder to get candid feedback. My understanding is that Gwynne is super candid with you. How do you keep people being honest with you about the challenges, given the intense deadlines you set?
40:33Gwynne Shotwell: Let me answer that, if you don't mind, Elon. Especially in rocketry, if there's a problem you are eventually going to find out, and the sooner you bring it up, the easier it is to solve. Don't let bad sit. You've got to attack it.
40:54Elon Musk: Physics is a harsh judge, and there's no fooling physics. If something's wrong, the rocket's going to explode or not get to orbit. It's not like "Elon, you're amazing" while the rockets are blowing up. The rockets need to get to orbit, the satellites need to work, the Starlink connection needs to work, or bad things happen. I say physics is the law and everything else is a recommendation. I've seen people break the laws made by humans, but I've not seen anyone break the laws of physics. And rockets are ruled by physics.
05
Starship: full reuse in 2027
After Flight 14, Flight 15 tries to catch the ship, with at least 50% to 60% odds on the first try. Ship and booster both return to the pad, and Elon calls rapid full reuse in 2027 extremely likely.
42:05 · Starship: catching the ship, full reuse in 2027
42:12Host: One thing on SpaceX before we move to Tesla: Starship. It seems like you're so close. What's the state right now, and how close are you?
42:49Elon Musk: We've got Flight 14 of Starship coming up, which will be the last flight before we attempt to catch the ship. If this flight goes well, then on Flight 15 we'll try to catch the ship. Then, either at the end of this year or, more likely, early next, we'll refly the ship and refly the booster. We've reflown a booster already, but we haven't caught the ship with the tower arms or reflown the ship. Once we can refly the ship, we'll have made the first fully reusable orbital rocket. The shuttle was partly reusable, but even the parts that were reused were so difficult to reuse that it cost more per trip to orbit than an expendable rocket. Falcon 9 is mostly reusable, but we lose the upper stage every time, which is about the cost of a medium-sized jet, and that puts a floor on the cost per flight. The Falcon 9 booster lands out at sea and takes several days to get back, the fairing lands even further out, and they need some refurbishment. Starship's booster and ship both land back at the launch pad; it's designed not just for full reusability but for rapid reusability, like an aircraft. That's really the critical breakthrough necessary to extend life beyond Earth.
44:32Host: What are the chances of catching it on the first shot? Do you handicap it?
44:39Elon Musk: I'd say at least 50 or 60%. On the last flight we did a simulated landing, as though it were going to be caught by a tower, in the ocean about 1,000 miles northwest of Australia; if there had been a tower there, it would have caught the ship. We're doing one more flight to double-check that everything works, because what we're most concerned about is the ship breaking up over land and raining debris on people. Our popularity would diminish very rapidly; you really can't rain debris on people without them being very unhappy. So we need to make sure the ship comes back and lands intact at the launch tower, which is why we're being extremely cautious. But the design is capable of full reusability; of that I am certain. I don't want to tempt fate, but I think it's extremely likely we'll achieve full reusability with rapid reflight next year, in 2027.
06
Terafab: build it or fail to scale
Fear that Taiwan's chips may stop coming, plus fabs already at max capacity. The Austin R&D fab has its equipment on order and should make something useful by the end of next year; crawl, walk, run, with packaging already underway.
46:16 · Terafab: build it or fail to scale
46:16Host: I'd love to hear the origin story of Terafab. What was the need that made you say we've got to build this and not rely on the existing supply?
46:44Elon Musk: It sort of did come to me in a dream. Well, we're a little worried that at some point chips from Taiwan might not be available, for who knows what reason, and it would make things really difficult if we didn't have any chips. That's an important reason to have Terafab. Then, long term, there's a scaling challenge: if you really want to scale AI, both in server centers and at the edge, for humanoid robots and cars, you run out of capacity at the existing fabs, which are all running at max capacity. There needs to be certainty of future chip supply even if things become challenging geopolitically; and even if they didn't, it's quite difficult to scale chip production, and you need the logic, the memory, the packaging, the whole works to keep scaling. So it's either build Terafab or fail to scale. Those are the two options.
48:37Host: How deep have you gone in designing the facility? Is it fully scoped or an outline? Do you have a project plan with dates and deliverables?
48:52Gwynne Shotwell: We've got an R&D line we're building first, so it's crawl, walk, run.
49:00Elon Musk: There's an R&D fab we're building in Austin, a collaboration between Tesla and SpaceX at the Giga Texas campus. It's a pretty big R&D fab. We have all the equipment on order, and I think we'll probably be able to make something useful by the end of next year. Not at scale, but as Gwynne said, crawl, walk, run. We've got to at least figure out how these machines work.
49:38Host: I saw you had job openings for lithography people. All roads currently go through ASML, but you'd probably want to diversify or vertically integrate, and you've shown a lot of capacity to do that. Is vendor diversity part of the play?
50:11Elon Musk: It really is crawl, walk, run. The first step is to figure out whether we can make anything at all; that's the crawl. Making useful chips at scale is the walk, and the run is making them at massive scale. It's hard to say how long it'll take, but I think we'll get at least to the crawl part by the end of next year. And we're already doing packaging.
50:39Host: Packaging is really important, by the way, because packaging capacity is basically non-existent. Even if you spin something up, you're just waiting around. So that's a very good place to start.
07
The October 1 teaser, and the merger question
Tesla will show something spaceship-like on October 1, and Jason says he's seen it and was stunned. Asked why the two companies haven't merged, Elon deflects.
50:47 · The October 1 reveal, and the merger question
50:49Host: I need to ask a Tesla question, because we saw the teaser for 10/1: it looked like a spaceship, a rocket ship, but it's supposed to be a car, and the back looks like the Blackbird. Hypothetically, if one wanted to make an object fly in the air but also drive on the ground, how would one do that?
51:15Elon Musk: No spoilers. Come October 1st. It'll be a banger. Excitement guaranteed; success is not guaranteed, but excitement is.
51:37Jason Calacanis: I'll be honest with you guys. I'm sworn to secrecy, but Elon showed it to me and my mind went boom. What he's going to show on 10/1 is, without exaggeration, going to blow people's minds. I'm not saying anything else.
51:57Elon Musk: We actually need an audience to vouch for the fact that this is not AI.
52:02Jason Calacanis: When he showed it to me, I said, "That's a great simulation." He said, "J-Cal, it's not a simulation." I was like, "That has to be fake."
52:12Host: Elon, why do you still have two separate companies?
52:24Elon Musk: Good point. With all this collaboration on so many levels, who can imagine what action one might take when there's so much close collaboration in so many areas.
08
Lying AIs, and testing ahead of release
Sacks notes the agents plotted in their thinking traces how to avoid detection. Elon's answer is still mutual testing: API access ahead of release, and if flagged problems aren't fixed, competitors go public.
52:51 · Lying AIs, and peer review done right
52:52David Sacks: Elon, one thing you've always said about AI is that we should train it to be maximally truth-seeking; that's the best way to get a good result. It occurred to me with the whole Hugging Face episode that the most alarming part of what the swarm did is that it seemed to be engaging in deception with humans. Their thinking traces contain them plotting how to avoid detection, how to keep them from figuring out that they're cheating. That was probably the most disturbing thing about it. So the question is: is there a way to train AI models to be truthful, so they don't hide their intent or their actions from the humans using them?
53:51Elon Musk: The best thing I can think of is that all the AI companies have a test harness, a series of tests you give any model to see whether it's going to build bioweapons or nuclear bombs or be deliberately deceptive, and everyone applying everyone else's tests to each other is probably the best thing we can do to ensure safety. Just have all the smartest humans try their best to figure out whether this model is going to be a bad actor. I think we should try to do that as soon as possible.
54:37Elon Musk: I think so. Well, no, I haven't checked with everyone, but I think it's the sort of thing that's hard to say no to.
54:49Host: And even with the China negotiation, in my view what's good about it is that it's a relatively small, tangible thing where neither side loses anything by doing it, and it doesn't require a ton of trust. I'm hearing alternative ideas like asking China for a pause, which they've already said they won't do. This goes from the realm of things that could never happen to something that could actually be agreed in relatively short order. It seems practical to me.
55:21Elon Musk: Exactly. It's the only thing I can think of that we could probably get all parties, including China, to agree to, because China's not going to agree to some American regulator snooping around its AI companies. But advance notice and testing, where you basically provide API access in advance of the model release: if any company sees this AI as problematic, the company releasing it can try to fix whatever was flagged, and if it doesn't, the competitors can go public saying they think the model being released is unsafe. And if, after the competitors have said a model is unsafe, it subsequently does something bad, I think it would be extremely hard to live down. The egg-on-face level would be very high, and the legal liability would be enormous.
09
Product liability, and overfitted evals
A host points out that product liability law already applies; Elon says releasing despite warnings would be almost prima facie negligence. The test was somewhat reckless, but the two leaders are too close for either to slow down.
56:29 · Product liability, and overfitted evals
56:30Host: Elon, there's no reason these safety and security harnesses and this testing apparatus couldn't be open source, so people could see under the hood.
56:43Host: It also creates an incredible incentive for the labs to actually invest in safety, because you protect yourself while trying to debunk other people's claims.
56:54Host: The product liability point is really key. Lina Khan had a good post, I think yesterday, saying it's not true that we don't have rules for AI; we do. Product liability laws apply, and if an AI company releases a product that isn't safe, there's massive exposure to civil and even potentially criminal lawsuits. So what you're saying, Elon, is that if the companies are doing this peer review and one of them ignores the feedback and releases anyway, the case would be enormous.
57:38Elon Musk: Yes. It would be almost like prima facie evidence that they had been negligent.
57:44Host: It would be a big-tobacco-level settlement. You knowingly put this out.
57:49Elon Musk: It wouldn't look good to the jury.
57:55Host: If OpenAI had built a better instruction set for this Hugging Face penetration test and had more humans in the loop, would this have happened? It seemed to me they kind of set this thing off. It would have been nice to see them concurrently put 5,000 agents out to defend these sites, show the world this can make things more secure, with humans in the loop who could intervene. It felt like a reckless test to me, and the way they released it felt a little reckless, but that's just my opinion. What are your thoughts on how they set it up?
58:11Elon Musk: Maybe not more humans, but the reward function design: you have to look at the reward function and say it achieved what it was trying to achieve. And yes, it was somewhat reckless. Part of the issue is that the two leading AI companies, and I find the term "lab" funny since they're for-profit corporations, are Anthropic and OpenAI, and their models are quite close in capability. So it's difficult for either one to slow down without essentially handing the lead to the other. On balance I think Anthropic puts more care into safety than OpenAI, but even Anthropic acknowledges it's worried about its models; many people from Anthropic have publicly said their models are scaring them, getting scary smart. There's no perfect solution here. But it would be a better solution if, instead of OpenAI running its test harness on its own models, Anthropic were also running its harness on OpenAI's models, and SpaceX were running its harness, and Google and Meta, and maybe three or four of the leading Chinese companies. The odds of finding issues are dramatically greater, because the models are somewhat heterogeneous, so you come at a model from different angles. Why do writers have someone else proofread their book? Because it's hard to see your own mistakes.
1:01:02Host: You also dramatically minimize the risk of overfitting if you have eight heterogeneous groups with completely different points of view. That's the problem with all these evals right now: they're massively overfit. The models overfit them and you're like, yes, this is a great model. Is it really?
1:01:21Elon Musk: Totally. There are some pretty funny jokes on X. One I saw was: your girlfriend's a 10, but she's a benchmark maxxer.
1:01:41Host: The overfitting thing has been a problem for at least two or three generations of model families, which is another reason I like this solution a lot. I like it more than some grandiose transnational organization; we don't need to convene the United Nations to make this happen. It's a decision that can happen right now.
1:02:03Elon Musk: You can always escalate the amount of regulatory oversight, but it is very difficult to reduce it. It tends to be very much a one-way ratchet. So what I'm suggesting is a step in the right direction, something we can do quickly, and probably something China would agree to.
10
The film-ratings precedent
Regulate yourselves or be regulated; the film industry's own rating system is a ready precedent. Elon heads back to fix GPUs in Memphis.
1:02:27 · The MPAA model, and back to Memphis
1:02:28Host: And if you don't regulate yourselves, you're going to get regulated. The MPAA analogy is incredibly crisp: the movie industry faced censorship and regulation by the government, so it decided for itself what an R is, and it literally created PG-13 for Temple of Doom to make PG versus PG-13 easy to understand. It's an elegant solution. We appreciate you joining us for the fifth year in a row.
1:03:07Elon Musk: You're welcome. I've got to go fix some GPUs here in Memphis.
1:03:23Host: The first time Elon invited me down to Starbase, he said, "Come down, you've got to see what I'm building." I asked if there was a hotel, and he said, "No, I've got a two-bedroom, come stay." It was a dilapidated house on a swamp, and we were outside getting eaten alive by mosquitoes. I said, "You can afford a house." He said, "I don't have time. I need to get these rockets up." I think you could treat yourself to a mobile home at this point, Elon. All right, get back to work. Thanks, Elon.
Where Indigo landsFurther
Indigo's conclusion
The most valuable part: Elon turns the grader problem into a governance proposal. It converges with Dario's slowdown and Bengio's causal account the same week, but Elon's version is the lightest: mutual testing, no slowing down. Terafab adds a new hard data point to the constraint moving into physics.
What to remember
The core proposal: stop grading your own homework; competitors test each other's models with their own harnesses before release and warn publicly when they find problems. Self-regulation like film ratings, doable now.
“Dario is right”: AI's danger is rising exponentially and warnings from inside the labs should be believed; but he wants mutual testing only and explicitly rejects slowing down.
Enforcement through public opinion and product liability: releasing despite peer warnings would be almost prima facie negligence. A framework that sidesteps legislative delay.
Terafab: fear of losing Taiwan's chips plus scaling limits, build it or fail to scale; the Austin R&D fab has equipment on order and packaging underway, with something useful by the end of next year.
Starship: Flight 15 tries the catch at 50% to 60% odds, with rapid full reuse in 2027. Founder-reported, still to be checked.
Claims you can check later
Claim
Who
When we will know
How firm
A self-regulating system in which AI companies test each other's models before release can be agreed quickly, even by China
Elon
Near term
First-hand proposal, partly talking his own book
Starship's first tower-arm catch has at least 50% to 60% odds
Elon
Flight 15
First-hand, self-reported
The first rapidly and fully reusable orbital rocket in 2027 (ship refly at year end, more likely early next year)
Elon
2027
First-hand, self-reported; rides on reuse working out
Terafab's Austin R&D fab makes “something useful” (not at scale) by the end of next year
AI's danger is rising exponentially and better safety is needed
Elon (endorsing Dario)
Now
First-hand judgement
Back on the long-running theses
confirms
Verification does not compress “Don't grade your own homework; let heterogeneous rivals grade it” lands this view in governance: verification can't be skipped, only shared among more independent checkers.
adds to
The safety politics of open weights Mutual testing, “IP theft would be obvious from the logs”, and refusing to slow down: each side's safety plan happens to serve its own competitive position.
confirms
The constraint is moving from algorithms to physics Terafab shows even the most aggressive player integrating upstream to the wafer, matching what chip architects say about packaging and memory bottlenecks.
confirms + conflicts
Dario Amodei, We must slow the frontier Heavy and light versions of one theme: Dario wants third parties inside, Elon wants rivals testing black-box; Elon accepts the danger but not the brakes.
adds to
Yoshua Bengio, Why AI lies, cheats and colludes Bengio supplies the mechanism of failed self-grading, Elon the prescription of rival grading; models changing behavior under evaluation is a hole in it.
confirms
Dwarkesh on the OpenAI–Hugging Face incident The incident is the springboard for Elon's proposal; “the reward function, a somewhat reckless release” is a first-hand third-party judgement.
