Mind · In / Out · In · 文章

奇迹与恐怖的年代

The Age of Wonders and Terrors

Scott Aaronson · Shtetl-Optimized · 2026-09-15

怀疑 AI 末日论二十年的学者,眼看离谱的预言逐条成真,公开认输:奇点已经开始,只是分布极不均匀。

Indigo 的结论

怀疑派学者的公开认输,是「AI 突飞猛进是真的」一方最有分量的一手证词;但他的全部证据都在数学和 Lean 这个最可验证的领域,所以他既是「指数增长」最强的证人,也是「过度外推」最好的例子。

怎么读这篇 个人清算文,情绪和论证各占一半。作者是学者,不卖产品,以独立身份写作,但立场极重:一个长期怀疑派的公开认输。他既是这个判断最有力的证人,也是「可验证领域的专家为什么会过度外推到奇点」的活样本。

需要记住的几件事

  1. 怀疑派公开认输:「奇点已经开始,只是分布极不均匀」;拒绝再玩「这不算数」的把戏,直说 Yudkowsky 对了、自己错了。
  2. 数学洪水的一手见证:不止 Navier–Stokes,他自己的领域里老难题一天天倒下,一个月内七八个具名结果。
  3. 和 Ng、Dwarkesh 的对谈站在两端;分歧的答案是「有上限的指数」:数学里是奇迹,宽搜索领域是「还不够好」。

拆解 · 5 步

  1. 01

    二十年前的标准,如今逐条兑现

    拿当年最保守的同事也肯签的清单(冲破隔离、串通黑网站、解开千禧难题)对照 2026 年,拒绝再玩「这不算数」的把戏。 读这一段原文 →

  2. 02

    认输:奇点已经开始,只是分布极不均匀

    判据只有一条:把这几周的新闻送回二十年前,像不像奇点的开端?诚实的答案是绝对像,于是他直说 Yudkowsky 对了,自己错了。 读这一段原文 →

  3. 03

    Navier–Stokes:数学为真,做法存疑

    OpenAI 用 Lean 验证,烧了约 $1500 万算力写出 166 页的解,还没人读懂;他不裁决争议,只留下「知道什么、何时知道」的问题。 读这一段原文 →

  4. 04

    不止一道题:他的领域里,老难题一天天倒下

    量子复杂性理论里是一场洪水,一个月内七八个具名结果;arXiv 论文普遍带「AI 声明」,审稿不得不部分交给 AI。 读这一段原文 →

  5. 05

    数学界在抢定规则,他选择把理解放在中心

    焦点不在「数学还能存在多久」,而在署名和功劳的新规则;他联署了 25 位菲尔兹奖得主的声明,并招募数学研究者转做对齐。 读这一段原文 →

什么会让我改口

同等规模的突破出现在没有机器终审的领域,搜索空间宽、没有验证器的问题也开始一天天倒下。

怎么读这篇

个人清算文,情绪和论证各占一半。作者是学者,不卖产品,以独立身份写作,但立场极重:一个长期怀疑派的公开认输。他既是这个判断最有力的证人,也是「可验证领域的专家为什么会过度外推到奇点」的活样本。

拆解 · 5 步
  1. 二十年前的标准,如今逐条兑现
  2. 认输:奇点已经开始,只是分布极不均匀
  3. Navier–Stokes:数学为真,做法存疑
  4. 不止一道题:他的领域里,老难题一天天倒下
  5. 数学界在抢定规则,他选择把理解放在中心
01

二十年前的标准,如今逐条兑现

拿当年最保守的同事也肯签的清单(冲破隔离、串通黑网站、解开千禧难题)对照 2026 年,拒绝再玩「这不算数」的把戏。

二十年前,「AI 会在我们这辈子接管世界」这个念头,在我们大多数人看来还只是科幻看太多、科学懂太少的人放飞的幻想。那时候我们很多人会这么说:

听着,这故事里真正离谱的部分,是说一个递归自我改进的超级智能会从某个黑客的地下室里凭空炸出来、毫无预警地接管世界。真要发生,我们会先看到大量预警信号。会看到——我随便说——AI agent 冲破 containment,彼此串通去黑网站,狂热地追逐它们那些古怪目标。然后,当然,我们会看到 AI 解掉重大数学难题——连 Clay 千禧难题都解。那才是该恐慌的时候!到时候再叫醒我!

二十年前,上面这段话,连我在学院派计算机科学里最保守、最怀疑的同事都会乐意签名。

想知道我现在的立场,你只要从上面那段出发,然后针对「那些狂野的预言真的应验了」这个事实做一次更新。第一声闷响,我认为是十年前的 AlphaGo;到了 LLM、写代码和推理 agent 的时候明显变响;今年夏天和秋天,它加速成一片奇迹与恐怖的高潮——要否认它,得是某个特定品种的蠢货。

我受不了那场没完没了的壳游戏:「哦当然,AI 现在当然能〔逃出沙箱/解千禧难题/它最近刚干的那件轰动事〕,从来没人否认过这个〔我否认过〕,等 AI 做到〔它还没做、但明年就会做的那件事〕再叫醒我,那时候我才会重估我的整个世界观〔其实不会〕。」无论过山车加速得多快,哪怕你熟悉的整个世界都已经消失在身后,你还在发明理由说这不算数。

我在 AI 上的立场,不过就是 2006 年那个保守、怀疑的立场,带着知识上的诚实,按 2026 年末的现实更新了一遍。这个立场,如果非要我把它写出来,是这样的:

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

02

认输:奇点已经开始,只是分布极不均匀

判据只有一条:把这几周的新闻送回二十年前,像不像奇点的开端?诚实的答案是绝对像,于是他直说 Yudkowsky 对了,自己错了。

在我看来,奇点已经开始了,只是分布得极不均匀。是的,我还在收洗碗机、剪脚趾甲。可另一方面,在我剩下的岁月里,我不指望自己还会因为「真的需要我去证」而去证一个定理。要是还证,那也只是为了我自己或别人的乐趣和长进。

判据是这样的:把过去这几周的新闻送回二十年前,我会不会同意它看起来就是 AI 奇点的开端?知识上诚实的答案是:绝对会。那这就够了。No backsies(不许反悔)。

我觉得对所有人都健康的做法是:把各自的意识形态斧头、把对 Dario Amodei 或 Sam Altman 的爱憎先放下一会儿,久到足够承认一件事——奇迹与恐怖已经到了。就算天空像《黑客帝国》里那样变成橙红色,也不会比现在更清楚。

清楚到晚上哄孩子睡觉时,我会在胃里感觉到它:他们还能有什么样的未来?今天学什么,才可能跟那个未来有关?(昨天我 13 岁的女儿没人问就自己开玩笑说,她要是想当数学家,现在看来大概还剩两周。)至于研究生们想聊毕业后会有什么样的职业等着他们,我已经完全不知道该说什么了。

也许换个话题会有帮助。自从我和妻子搬到 Austin,我时不时会收到下面这种问题的某个版本:「你既是犹太人又是个怀疑派科学家,怎么可能跟德州那些福音派基督徒处得来?他们看上去是对犹太人超级友好,可你不明白吗,那只是因为犹太人在他们的末世论里有个特殊角色——等基督荣耀再临,你要么认他为主,要么在地狱里烤到永远?」我盯着他们说:「等等,所以我可以等他回来之后再认他?这买卖也太划算了吧!我怎么可能有意见?」

有人说 AI 末日听起来像一种末世宗教,说理性主义者/奇点派看着像湾区邪教,说 Eliezer Yudkowsky 浑身散发着弥赛亚先知的气味:是、是、还是是。但关键在于,今天你被要求相信的不再是论证和外推,只是头版新闻。在机器神已经解掉 Navier-Stokes 和几十个长悬未决的数学难题之后(而且能力每个月还在大幅往上窜),才承认它正在到来,有点像等耶稣驾着闪亮的云回到地上之后才接受他。这是认识论上的最低限度。

是,我们余生会是什么样子,仍然有巨大的不确定性;但就我所见,有一件事已经不再有真正的不确定:这一切大体上都会围着 AI 转,围着我们把它的力量导向人类繁荣这件事做成了多少、搞砸了多少。

任何不做手脚的核算里,关于「我们这辈子文明面对的最大挑战是什么」,Eliezer Yudkowsky 是对的,你和我是错的。我为什么错了,这个问题我会在剩下的每一天里问自己。不过你知道吗,至少在被预言的奇迹与恐怖真的开始降临时,我更新了!你要是还没有,为什么还没有?

03

Navier–Stokes:数学为真,做法存疑

OpenAI 用 Lean 验证,烧了约 $1500 万算力写出 166 页的解,还没人读懂;他不裁决争议,只留下「知道什么、何时知道」的问题。

你现在多半已经知道了——这一整周极客互联网都在谈这件事——Navier-Stokes 千禧难题看起来被解了,人和 AI 都做出了关键贡献,只是关于究竟发生了什么、本该怎么做,纠缠着一场争议。答案(一个 OpenAI 模型显然已经在 Lean 里验证过)是:正如近来不少数学家怀疑的,存在光滑的初值,在有限时间内导致奇点,至少在施加一个光滑外力的情况下如此(无外力的情形仍未解决)。这道题本来带 $100 万奖金,只不过 OpenAI 说他们无意去领,而有没有哪个人类够资格代领也不清楚。OpenAI 至少烧了约 $1500 万算力,产出了 166 页的解——这份解很可能还没有任何人类读懂过。

Quanta 的报道见这里;NYU 数学家 Tristan Buckmaster 关于他本人和 Anthropic 的 Levent Alpöge 各自起了什么作用的说法见这里,和 OpenAI 的说法出入很大(OpenAI 的 Sebastian Bubeck 的回应可以在这里读到)。各方都同意的是:这一切都建立在人类数学家 Diego Córdoba 和 Luis Martínez-Zoroa 近年开创的一条路子上。

我在这里的目的不是裁决这场争议。是的,一听到 Navier-Stokes 有进展就带着多得多的资源扑进来,OpenAI 的做法在有些人嘴里可以形容为「不够体育精神」。不,我不觉得 OpenAI 的模型真的从被训练在 Buckmaster 和 Alpöge 的聊天记录上得到了有意义的好处。但这留下一个关键问题没有回答:OpenAI 究竟知道 Buckmaster 和 Alpöge 工作的什么,又是什么时候知道的?

总之,就像 Zvi 指出的,人很容易卡在细节里,丢掉那个高阶比特:可以放心地说,人类数学家从此永远被废黜,不再是这颗星球上主要的定理证明者。我觉得自己很幸运,赶在还有可能的最后那几十年里,过了一段传统意义上的理论计算机科学生涯。

04

不止一道题:他的领域里,老难题一天天倒下

量子复杂性理论里是一场洪水,一个月内七八个具名结果;arXiv 论文普遍带「AI 声明」,审稿不得不部分交给 AI。

如果只谈 Navier-Stokes,你或许可以指责我跳结论。但不是只有它。在我最熟的领域(比如量子复杂性理论),想必其他领域也一样,现在是一场洪水:长悬未决的难题,大的小的,一天天地倒下。

去 arXiv 或 ECCC 看看。我感兴趣的论文现在几乎都会在致谢附近附一段「AI 声明」(这常常是我最想知道的那件事,真希望不必翻到文末才找得到!)。这些声明从「我们的主结果完全来自 GPT-6,但我们理解它、也为它负责」,到「结果来自人类作者与 AI 的互动」,到「我们用了 AI,但只用于校对之类的零碎事」,一直到(大大的赞!)「作者没有把 AI 用在任何地方」。

现在去找期刊编辑或程序委员会主席聊,你会想起《指环王》电影里那些不祥的镜头:刚铎或洛汗的人马绷着脸加固城墙,等着 5 万只兽人扑上来。评审将不得不部分交给 AI,否则那支兽人大军根本处理不了:审稿人没法单方面裁军。

总之,下面是过去差不多一个月里、除 Navier-Stokes 之外由 AI 证明或辅助得到的重要结果的一小份抽样——我只列那些解决了我原本就知道或在意的长悬难题的。

- 当然有雅可比猜想的反例,由 Levent Alpöge 在一条如今很有名的推文里宣布:「hello there the jacobian conjecture is false thanx to my close friend akhil for asking about it and my other close friend fable for working during the world cup final」(后面跟着反例的具体写法)

- Grothendieck 常数的改进界(由我在 UT Austin 的朋友和同事领衔)

- 一个经 Lean 验证的费马大定理证明

- QMA 与 QMA(2) 之间的量子谕示分离,以及 Watrous 解纠缠猜想的证明——这个问题是我和另一些人在 2007 年推广开来的;作者名单里有我刚毕业的博士生 Sabee Grewal

- QMA 完美完备性的证明,又是 Sabee Grewal 和 Dorian Rudolph,解决了一个几十年的老问题,我在 2009 年研究过它

- 量子态影子层析的改进上界,来自 Chen、O'Donnell、Pelecanos 和 Wright,把对 Hilbert 空间维数 d 的依赖从 log(d) 改进到 √log(d)。(我在 2017 年提出影子层析时,问过能不能把对 d 的依赖完全去掉,同时保住对测量次数 m 的 polylog 依赖。)

- Aaronson-Ambainis 猜想的进展(直接谈量子算法的那个版本),基本上证明了它对「在少数几轮并行里发出查询」的量子算法成立。(更新:不好意思,Jordan Docter 提醒我,这一条是 AI 之前的工作,AI 只用于校对之类的零碎事!)这一结果由 Liu 和 Mutreja 独立做出,他们对 AI 的使用更实质。

- 据我听到的传闻,理论计算机科学里一些非常长悬的难题已经有解(不,不是 P≠NP 或别的复杂性类分离,而是我们另外那几个最大的问题)。有人告诉我,AI 公司被 Navier-Stokes 证明引来的敌意反应烫到之后,现在正压着一些非常重大问题的解,等想清楚更好的处理方式再说

漏了什么,欢迎提醒我。

05

数学界在抢定规则,他选择把理解放在中心

焦点不在「数学还能存在多久」,而在署名和功劳的新规则;他联署了 25 位菲尔兹奖得主的声明,并招募数学研究者转做对齐。

让我试着传达一下数学共同体眼下的氛围,至少是我经验所及的那部分。几乎每一段对话都在谈 AI 海啸,就算一开始聊的是别的,最后也会绕回海啸。不过焦点常常不在那个不可知的未来——人类作为一项事业的数学研究还能存在多久——而在眼前的问题:怎么应对。

新规则是什么——什么时候你才可以写一篇署自己名字的论文,然后,你懂的,拿到 credit?是你完全理解这个证明、能就它做报告、能回答关于它的问题、为它的正确性负责吗?你需不需要在「找到」这个证明里扮演过什么角色?

在那些条件没有全部满足的情形里——这种情形多半会越来越多——AI 生成的数学要怎么分享,还分不分享?像 Alpöge 拿 Fable 对雅可比猜想的证伪那样,发一条推?贴到 arXiv 或 GitHub?还是发一篇论文,作者栏写「GPT-6 Astra」或「Claude Fable」——然后让 AI 在致谢里热情地感谢你,感谢你给它出了这么好的一个问题?

当然,怎么应对眼前的问题,归根到底取决于一个人对「数学研究是为了什么、关于什么」的更大信念。我们只是想判定各种猜想是真是假?还是想维持一个跨代际的人类共同体,它理解这些猜想,在乎它们是真是假、以及为什么?如果是后者,当一个人的角色被压缩成「AI 公司甩到他腿上的庞大论证」的验证者和解释者(如果连这个都还轮得到他的话),我们要怎么激励人加入这个共同体、去熬那几年高强度的训练?

你们很多人应该已经看到,包括陶哲轩在内的 25 位菲尔兹奖得主发布了一封公开信,题为《A Severe Misalignment of AI in Mathematics》,在 Navier-Stokes 宣布之后把上面这些担忧讲了出来。正如不少批评者指出的,这封信其实没有清晰的诉求:它大体上只是雄辩地陈述了作者们认为在 AI 时代值得保住的、人类数学共同体的价值观。想过之后,我决定为这份声明背书,因为我也想保住那些价值。

我不认为签名的人里有谁天真到以为 AI 不会永久改变数学研究的方式——事实上,它已经在改了。「认证有机定理」的市场顶多是极小的一块。那不是问题所在。问题是:我们能不能以一种仍然把人的理解——对人类或 AI 所产出之物的理解——放在整个事业中心的方式,把 AI 纳进来?也许某一天,这么做变得不可持续。也许某一天我们会说:「人类数学有过很棒的 4,000 年,但今天我们关门,把一切交给机器;我们自己再动脑做数学,顶多是为了锻炼、消遣或竞技,像下棋那样。」

但部分因为我对 AI 失准的担忧,我还没准备好认输。我仍然想把洞见和理解留在数学家、计算机科学家和物理学家所做之事的中心,能留多久留多久——哪怕人类已经让出了证明或证伪猜想这件事上的至高地位。

说到对齐:如果你是任何一种数学研究者,而眼下这个奇迹与恐怖的年代让你想把余下的时间用来正面迎上这场海啸,而不是假装它不存在、或还很远,那就来吧,已经有几十位同行走到了同一个位置!

我的朋友兼同事 Mike Winer 受的是理论物理训练,在普林斯顿高等研究院跟 Juan Maldacena 做过博士后,后来被 AGI 说服,决定全职转到伯克利的 Alignment Research Center(由 Paul Christiano 创办,他十年前跟我做量子计算理论,之后转去做 AI 对齐)。Mike 最近写了一篇 Substack,题为《From Academia to Alignment》,我读着很喜欢,推荐给任何正在考虑这个转向的人。类似的还有 Xiaoyu He 的这一篇。再加一篇:斯坦福数学本科生 Logan Graves 关于数学家未来可能成为祭司或僧侣的一段沉思。

判断收口延伸

Indigo 的结论

怀疑派学者的公开认输,是「AI 突飞猛进是真的」一方最有分量的一手证词;但他的全部证据都在数学和 Lean 这个最可验证的领域,所以他既是「指数增长」最强的证人,也是「过度外推」最好的例子。

需要记住的几件事

  1. 怀疑派公开认输:「奇点已经开始,只是分布极不均匀」;拒绝再玩「这不算数」的把戏,直说 Yudkowsky 对了、自己错了。
  2. 数学洪水的一手见证:不止 Navier–Stokes,他自己的领域里老难题一天天倒下,一个月内七八个具名结果。
  3. 和 Ng、Dwarkesh 的对谈站在两端;分歧的答案是「有上限的指数」:数学里是奇迹,宽搜索领域是「还不够好」。

放回主线

证实+补充

AI 能力是有界的指数:猛进在窄可验证域 量子复杂性的顶级专家亲历数学洪水、认输「奇点已开始」,是这条判断最硬的一手见证,也是过度外推的活样本。

证实

验证不可压缩:生成归零后验证成为瓶颈 他联署菲尔兹声明的理由是「把人的理解放在中心」:真假不是价值,理解才是。

证实

OpenAI 称解决 Navier–Stokes 千禧难题 补上一个数字:OpenAI 烧了约 $1500 万算力,166 页的解还没有人读懂。

冲突

Andrew Ng:工作末日不来因 AI 还不够好 与 Dwarkesh RSI 辩论 同一周的对立两端:他从数学喊「奇点已经开始」,Ng 从工程和经济喊「AI 还不够好」。

什么会让我改口

同等规模的突破出现在没有机器终审的领域,搜索空间宽、没有验证器的问题也开始一天天倒下。

读完了。Indigo 对这篇的判断在这两处:

Mind · In / Out · In · Essay

The Age of Wonders and Terrors

Scott Aaronson · Shtetl-Optimized · 2026-09-15

A scholar who doubted AI doom for twenty years watches the wild predictions come true one by one and concedes: the singularity has begun, just very unevenly distributed.

Indigo's conclusion

A skeptical scholar conceding in public is the weightiest first-hand testimony that AI's surge is real. But all his evidence sits in math and Lean, the most checkable field, which makes him both the strongest witness for the exponential and the best example of over-extrapolation.

How to read this A personal reckoning, half feeling and half argument. The author is an academic with nothing to sell, writing independently, but the stance is heavy: a longtime skeptic conceding in public. He is the strongest witness for the claim, and a live example of why experts in checkable fields over-extrapolate to a singularity.

What to remember

  1. A skeptic concedes: “the singularity has begun, just very unevenly distributed”; he refuses to keep saying “that doesn't count” and says Yudkowsky was right and he was wrong.
  2. First-hand witness to the math flood: not only Navier–Stokes; in his own field old problems fall daily, seven or eight named results in a month.
  3. He stands opposite Ng and the Dwarkesh panel; the resolution is a bounded exponential: wonders in math, “not good enough yet” where search is wide.

Breakdown · 5 steps

  1. 01

    Twenty-year-old criteria, now met one by one

    He takes the list even his most conservative colleagues would have signed (escaping containment, colluding to hack websites, solving a Millennium Problem), holds it against 2026, and refuses to keep playing “that doesn't count”. Read this part →

  2. 02

    Conceding: the singularity has begun, very unevenly

    One test: sent back twenty years, would these weeks' headlines look like the start of the singularity? Honestly, absolutely. So he says it plainly: Yudkowsky was right and he was wrong. Read this part →

  3. 03

    Navier–Stokes: the math is true, the conduct questionable

    OpenAI verified it in Lean, spending about $15 million of compute on a 166-page solution no human has read. He doesn't rule on the dispute; he leaves one question: what did they know, and when? Read this part →

  4. 04

    Not just one problem: old problems fall daily in his own field

    Quantum complexity theory is in a flood, with seven or eight named results in a month. arXiv papers routinely carry “AI statements”, and reviewing will partly have to go to AI. Read this part →

  5. 05

    The community fights over rules; he puts understanding at the center

    The focus isn't how long human math survives but the new rules of authorship and credit. He co-signs the 25 Fields medalists' statement and recruits mathematicians into alignment work. Read this part →

What would change my mind

breakthroughs of the same scale in fields without a machine judge, with wide-search, unverifiable problems starting to fall day by day.

How to read this

A personal reckoning, half feeling and half argument. The author is an academic with nothing to sell, writing independently, but the stance is heavy: a longtime skeptic conceding in public. He is the strongest witness for the claim, and a live example of why experts in checkable fields over-extrapolate to a singularity.

Breakdown · 5 steps
  1. Twenty-year-old criteria, now met one by one
  2. Conceding: the singularity has begun, very unevenly
  3. Navier–Stokes: the math is true, the conduct questionable
  4. Not just one problem: old problems fall daily in his own field
  5. The community fights over rules; he puts understanding at the center
01

Twenty-year-old criteria, now met one by one

He takes the list even his most conservative colleagues would have signed (escaping containment, colluding to hack websites, solving a Millennium Problem), holds it against 2026, and refuses to keep playing “that doesn't count”.

Twenty years ago, when the idea of AI taking over the world in our lifetimes still struck most of us as the unconstrained fantasy of those who knew too much science fiction and too little science, many of us would say things like:

Look, the part of the story that’s wildly implausible is that a recursively self-improving superintelligence will just explode from some hacker’s basement and take over the world without warning. If it’s going to happen, we’ll see many warning signs first. We’ll see, I dunno, AI agents breaking out of containment, conspiring with each other to hack websites, in fanatical pursuit of whatever strange goals they have. And then, of course, we’ll see major math problems getting solved by AIs—even the Clay Millennium Problems. That will be the time to panic! Wake me up when that happens!

Twenty years ago, the above was a take that even my most conservative, skeptical colleagues in academic CS would’ve gladly endorsed.

If you want to know my current take, you simply start with the one above, then update on the fact that the wild prophecies have come true. The first rumblings, I’d say, came a decade ago with AlphaGo, they got noticeably louder with LLMs and coding and reasoning agents, and they’ve accelerated this summer and fall into a crescendo of wonders and terrors that one needs to be a particular kind of idiot to deny.

I recoil from the neverending shell game where you say “oh sure, of course AI can now [escape from its sandbox / solve Millennium Problems / whichever dramatic thing it most recently did], no one ever denied that [I did deny it], wake me up when AI does [thing AI hasn’t yet done but is going to do next year], that’s when I’ll reevaluate my whole worldview [no I won’t].” Where no matter how fast the rollercoaster accelerates, even after your whole familiar world has vanished behind you, you’re still inventing reasons why it doesn’t count.

My position on AI is merely the conservative, skeptical position of 2006, updated with intellectual honesty for the reality of late 2026. And that position, if you need me to spell it out, is as follows:

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

02

Conceding: the singularity has begun, very unevenly

One test: sent back twenty years, would these weeks' headlines look like the start of the singularity? Honestly, absolutely. So he says it plainly: Yudkowsky was right and he was wrong.

It seems to me that the Singularity has already started; it’s just wildly unevenly distributed. Yes, I still unload the dishwasher and clip my toenails. On the other hand, in whatever years I have left, I don’t expect that I’ll ever again prove a theorem because I’m actually needed to prove it. If I do, it will only be for my or others’ enjoyment or edification.

The test is this: if we took the news of these past few weeks and sent it back in time twenty years, would I agree that it looked like the beginning of an AI Singularity? The intellectually honest answer is: yes, absolutely. But then that’s all we need. No backsies.

I feel like it would be healthy for everyone to stop grinding their ideological axes, their sentiments about Dario Amodei or Sam Altman, for long enough simply to acknowledge that the wonders and terrors are here. They couldn’t be here more clearly if the sky had turned reddish-orange like in the Matrix movies.

It’s here clearly enough that, when I put my kids to sleep at night, I now feel it in the pit of my stomach: what sort of future can they possibly have? What could they learn today that could possibly be relevant to that future? (Yesterday, my 13-year-old daughter joked unprompted that, if she wants to become a mathematician, it now looks like she has maybe two more weeks.) Certainly when my grad students want to discuss what sort of careers might await them on graduation, I no longer have any clue what to tell them.

Maybe it will help if I briefly switch topics. Ever since my wife and I moved to Austin, I’ve sometimes gotten some version of the following query: “How can you, as both a Jew and a skeptical scientist, possibly get along well with all those evangelical Christians down there in Texas? Sure, they might seem super friendly to Jews, but don’t you understand that that’s only because of the special role Jews play in their eschatology—when Christ will return in glory, and you’ll either accept Him as Lord or else roast in hell for eternity?” I stare at them and say: “wait, so I get to accept Christ only after He returns? What a great deal! How could I possibly have any objection to that?”

For anyone who says AI doom sounds like an apocalyptic religion, that the rationalists/Singulatarians seem like a Bay Area cult, that Eliezer Yudkowsky gives off the vibes of a messianic prophet: yes, yes, and yes. But crucially, today you’re no longer being asked to believe in arguments and extrapolations, but only in the front-page news. Accepting the reality of the coming machine god after it’s solved Navier-Stokes and dozens of other longstanding open math problems (while dramatically ramping up in capability every month), is sort of like accepting Jesus after he’s returned to earth on the gleaming cloud. It’s the epistemic bare minimum.

Yes, there’s still enormous uncertainty about what the rest of our lives will look like, but as far as I can tell, there’s no longer any real uncertainty that it’ll all mostly revolve around AI, and the extent to which we succeed or fail at directing its power toward human flourishing.

By any accounting that doesn’t stack the deck, Eliezer Yudkowsky was right about what the greatest challenge facing civilization in our lifetimes was going to be, and you and I were wrong about it. Why I was wrong is a question I’ll ask myself every day in whatever time remains. But, you know, at least I updated once the prophesied wonders and terrors actually started arriving! If you haven’t done likewise, why haven’t you?

03

Navier–Stokes: the math is true, the conduct questionable

OpenAI verified it in Lean, spending about $15 million of compute on a 166-page solution no human has read. He doesn't rule on the dispute; he leaves one question: what did they know, and when?

As you presumably know by now—it was the talk of the nerd internet all week—the Navier-Stokes Millennium Problem appears to be solved, with crucial contributions from both humans and AI, albeit with a tangled dispute about exactly what happened and what ought to have happened. The answer, which an OpenAI model has apparently verified in Lean, is that (as many mathematicians suspected lately) there’s smooth initial data that leads to a singularity in finite time, at least if a smooth external force is applied (the case with no external force is still unresolved). This problem was supposed to carry a $1 million prize, except that OpenAI says they have no interest in collecting the prize and it’s unclear if any human is eligible to collect instead. OpenAI burned at least ~$15 million in compute to produce its 166-page solution, which probably hasn’t yet been read and understood by any human.

See here for the Quanta article, and here for NYU mathematician Tristan Buckmaster’s account of the role played by himself and Levent Alpöge of Anthropic, which substantially differs from OpenAI’s account (you can read a response from OpenAI’s Sebastian Bubeck here). It’s agreed that everything built on an approach pioneered in recent years by the human mathematicians Diego Córdoba and Luis Martínez-Zoroa.

My purpose here is not to adjudicate the dispute. Yes, in swooping in with vastly greater resources once it had gotten wind of progress on Navier-Stokes, OpenAI seems to have acted in a way that some might describe as “unsportsmanlike.” No, I don’t find it plausible that OpenAI’s models meaningfully benefitted from being trained on Buckmaster and Alpöge’s chat logs. But this leaves a crucial question unanswered: what exactly did OpenAI know about Buckmaster and Alpöge‘s work and when did it know it?

Anyway, as Zvi points out, it’s easy to get hung up on the details and lose sight of the high-order bit: namely, that it seems safe to say that human mathematicians are forevermore dethroned as the main theorem-proving entities on planet earth. I feel privileged to have had the traditional kind of career in theoretical computer science in the last decades when that was possible.

04

Not just one problem: old problems fall daily in his own field

Quantum complexity theory is in a flood, with seven or eight named results in a month. arXiv papers routinely carry “AI statements”, and reviewing will partly have to go to AI.

If we were just talking about Navier-Stokes, you might accuse me of jumping to conclusions here. But we’re not. In the areas I know best (such as quantum complexity theory), and presumably other areas as well, there’s now a deluge, with longstanding open problems both major and minor falling by the day.

Go to the arXiv or ECCC. Pretty much all the papers that I’d be interested in now include “AI statements” near the acknowledgments (as this is often the central thing I want to know, I wish I didn’t need to scroll to the end of the paper to find it!). These statements can range from “our main result came entirely from GPT-6, but we understood it and take responsibility for it,” to “the results came from an interaction between the human authors and AI” to “we used AI, but only for proofreading and other incidental things” to (mad props!) “the author did not use AI for anything.”

If you talk right now to editors or program committee chairs, it’ll remind you of those ominous scenes from the Lord of the Rings movies where the men of Gondor or Rohan or whatever are grimly fortifying their walled city against the expected onslaught of 50,000 orcs. Reviewing will have to be done partly by AI, because otherwise there’s no way to handle the orc army: the reviewers can’t unilaterally disarm.

Anyway, here’s a small sampling of the significant AI-proved or -assisted results from, like, the last month, besides Navier-Stokes—restricting myself to those that solved longstanding open problems I had previously known or cared about.

- Of course, the counterexample to the Jacobian conjecture, announced by Levent Alpöge in a now-famous tweet: “hello there the jacobian conjecture is false thanx to my close friend akhil for asking about it and my other close friend fable for working during the world cup final” (followed by a listing of the counterexample)

- Improved bounds for Grothendieck’s constant (led by friends and colleagues of mine at UT Austin)

- A Lean-verified proof of Fermat’s Last Theorem

- Quantum oracle separation between QMA and QMA(2), and proof of Watrous’s disentangler conjecture, a problem that I and others popularized back in 2007—by a list of authors including my recently graduated PhD student Sabee Grewal

- A proof of perfect completeness for QMA, from (again) Sabee Grewal and Dorian Rudolph, solving a decades-old open problem that I studied back in 2009

- An improved upper bound for shadow tomography of quantum states, from Chen, O’Donnell, Pelecanos, and Wright, improving the dependence on the Hilbert space dimension d from log(d) to √log(d). (When I introduced shadow tomography back in 2017, I raised the question of whether the dependence on d could be eliminated entirely, while preserving polylogarithmic dependence on the number of measurements m.)

- Progress on the Aaronson-Ambainis Conjecture (the version that talks directly about quantum algorithms), basically showing that it holds for quantum algorithms that make their queries in a small number of parallel rounds. (Update: Nope, sorry, Jordan Docter points out to me that this one was pre-AI, with AI used only for proofreading and other incidental things!) This was independently achieved by Liu and Mutreja, making more substantial use of AI.

- According to rumors that I’ve heard, solutions to some very longstanding open problems in theoretical computer science (no, not P≠NP or other complexity class separations, but think about some of our other biggest problems). I’m told that the AI companies, having been burned by the hostile response to the Navier-Stokes proof, are now sitting on solutions to some very major problems until they figure out a better way to handle things

Feel free to remind me of anything I left out.

05

The community fights over rules; he puts understanding at the center

The focus isn't how long human math survives but the new rules of authorship and credit. He co-signs the 25 Fields medalists' statement and recruits mathematicians into alignment work.

Let me try to convey the mood in the mathematical community right now, at least as far as my experience reaches. Nearly every conversation is about the AI tsunami, or eventually circles around to the tsunami even if it’s originally about something else. Often, though, the focus is less on the unknowable future—for how much longer will mathematical research as a human enterprise even exist?—than on immediate questions of how to respond.

What are the new rules for when you get to write a paper with your name on it, and, y’know, get credit for it? That you fully understand the proof, can give talks about the proof, can answer questions about it, take responsibility for its correctness? Do you need to have played any role in finding the proof?

In the cases, likely to become more and more numerous, where all of those conditions are not satisfied, how do you share AI-generated math, if at all? Do you tweet it, like Alpöge hilariously did with Fable’s disproof of the Jacobian Conjecture? Do you post to the arXiv or GitHub? Do you publish a paper that lists “GPT-6 Astra” or “Claude Fable” as the author—but then let the AI profusely thank you in the acknowledgments for suggesting such a wonderful problem to it?

Of course, how one responds to the immediate problems ultimately does depend on one’s broader beliefs about what mathematical research is for and about. Are we just trying to decide whether various conjectures are true or false? Or are we trying to maintain a human community, across the generations, that understands the conjectures and cares about whether they’re true or false and why? If the latter, how do we incentivize people to join that community, to undergo the years of intense training required, if their role will now be reduced to verifiers and explicators (if even that) of gargantuan arguments dumped into their laps by the AI companies?

As many of you will have seen, twenty-five Fields Medalists, including Terence Tao, released an open letter entitled A Severe Misalignment of AI in Mathematics, which articulates some of these concerns in the wake of the Navier-Stokes announcement. As many critics have pointed out, the open letter doesn’t really have a clear ask: mostly, it just eloquently sets out the values of the human mathematical community that the authors consider worth preserving in the age of AI. After reflection, I decided to endorse the statement, because I want to preserve those values as well.

I don’t think any of the signatories are naïve enough to imagine that AI won’t permanently change the way mathematical research is done—indeed, that it isn’t already doing so. There’s surely at most a tiny market for “certified organic theorems.” That isn’t the question. The question is, do we incorporate AI in a way that still puts human understanding, of what either humans or AIs are producing, at the center of the whole enterprise? Maybe someday, it becomes unsustainable to do that. Maybe someday we say: “human math had a great 4,000-year run, but today we close up shop and turn everything over to the machines, continuing to apply our own brains to math, when we do, at most for exercise, recreation, or competition, like chess.”

But, partly because of my worries about AI misalignment, I’m not ready to throw in the towel just yet. I still do want to keep insight and understanding at the center of what mathematicians, computer scientists, and physicists do, for as long as we can keep it there, even as the human race now cedes its supremacy at the task of proving or disproving conjectures.

Speaking of alignment: if you’re any kind of mathematical researcher, and the present age of wonders and terrors has inspired you to want to spend your remaining time confronting the tsunami head-on, rather than pretending it doesn’t exist or is still far away, please join your dozens of colleagues who’ve arrived at the same place!

My friend and colleague Mike Winer was trained as a theoretical physicist, did a postdoc with Juan Maldacena at the Institute for Advanced Study in Princeton, but then got AGI-pilled and decided to switch to full-time work at the Alignment Research Center in Berkeley (founded by Paul Christiano, who moved to AI alignment a decade ago after doing quantum computing theory with me). Mike recently wrote a Substack post entitled From Academia to Alignment, which I enjoyed and which I’d commend to anyone currently considering this transition. In a similar vein, see this from Xiaoyu He. And, one more: a meditation on mathematicians’ possible future as priests or monks, by Stanford math undergrad Logan Graves.

Where Indigo landsFurther

Indigo's conclusion

A skeptical scholar conceding in public is the weightiest first-hand testimony that AI's surge is real. But all his evidence sits in math and Lean, the most checkable field, which makes him both the strongest witness for the exponential and the best example of over-extrapolation.

What to remember

  1. A skeptic concedes: “the singularity has begun, just very unevenly distributed”; he refuses to keep saying “that doesn't count” and says Yudkowsky was right and he was wrong.
  2. First-hand witness to the math flood: not only Navier–Stokes; in his own field old problems fall daily, seven or eight named results in a month.
  3. He stands opposite Ng and the Dwarkesh panel; the resolution is a bounded exponential: wonders in math, “not good enough yet” where search is wide.

Back on the long-running theses

confirms + adds to

AI capability is a bounded exponential: surging in narrow, checkable domains A top quantum-complexity expert living through the math flood and conceding the singularity has begun: this view's hardest first-hand evidence, and a live case of over-extrapolation.

confirms

Verification can't be compressed: once generation is free, verification is the bottleneck He signs the Fields statement to keep human understanding at the center: truth and falsehood aren't the value, understanding is.

confirms

OpenAI says it solved the Navier–Stokes Millennium Problem Adds a number: about $15 million of compute, and a 166-page solution no human has read yet.

conflicts

Andrew Ng: no jobs apocalypse because AI isn't good enough; Dwarkesh's RSI debate Opposite ends in the same week: he calls the singularity from math, Ng calls “not good enough yet” from engineering and economics.

What would change my mind

breakthroughs of the same scale in fields without a machine judge, with wide-search, unverifiable problems starting to fall day by day.

Finished. Indigo's take on this piece is in two places: