AI 与机器学习2025arxiv.org
推理模型推理得很好，直到它们不再如此
Reasoning models reason well, until they don't
进阶 · 约 45 分钟读完前置知识：大型语言模型基础、推理模型（LRM）概念、图遍历基础
导读
论文研究大型推理模型（LRM）在推理复杂度上升时是否仍能可靠推理。作者构建了推理复杂度可调的 DeepRD 数据集，把问题建模为图并依据遍历复杂度控制难度，发现现有模型在复杂度足够高时表现会急剧下降。
HN 关注点：“推理模型”是否真的会推理，还是只是同一 LLM 的提示链，以及新基准的复杂度设定和结论能否随规模扩大、RL 与工具使用而改变。
HN 评论精选
这几条评论里：支持 1 质疑 2 补充 1
支持模型擅长复述已知内容、不擅推导，且不会承认不知道，容易自信编造。
我觉得模型对“自己知道什么”掌握得还不错，但一旦超出这一点，进入“从已知内容能推理出什么”的范围，它们就严重缺乏这种能力。它们擅长重复训练数据，而不是对数据思考。我发现的问题是，它们随后不会停下来，也不会说“我不知道”（除非被明确提示），而是直接编造，并且用同样自信的语气表达。
My_Name查看英文原文
I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confidence.
质疑质疑 AI 的失败案例并不比普通人底层 10% 的失败更特别。
我还没见过哪个 AI 失败的任务，是人群中最差 10% 也不会失败的。
anal_reactor查看英文原文
I'm yet to see a task that AI fails at that bottom 10% of population wouldn't also fail at.
质疑认为“推理模型”只是 LLM 链式提示，不构成真正的推理能力突破。
这个结果并不意外（不意外也仍值得研究）。所谓 LRM 就是 LLM；不存在“推理模型”，本质上只是让 LLM 写出比人类更好的提示，再把它发回给同一个 LLM。尽管 Amodei 和 Altman 想让华尔街相信，他们并没有通过按顺序运行两个不同提示来回答用户问题，就突然解锁了 LLM 的推理能力。真正令人惊讶的是，既然推理模型和非推理模型本质相同，前者居然还能有任何提升。
iLoveOncall查看英文原文
> [...] recent studies show that transformers and LLMs fail catastrophically once reasoning problems exceed modest complexity. We revisit these findings through the lens of large reasoning models (LRMs) -- LLMs fine-tuned with incentives for step-by-step argumentation and self-verification This was the obvious outcome of the study (don't get me wrong, obvious outcomes are still worth having research on). "LRMs" *are* just LLMs. There's no such thing as a reasoning model, it's just having an LLM write a better prompt than the human would and then sending it to the LLM again. Despite what Amodei and Altman want Wall Street to believe, they did not suddenly unlock reasoning capabilities in LLMs by essentially just running two different prompts in sequence to answer the user's question. The truly amazing thing is that reasoning models show ANY improvement at all compared to non-reasoning models, when they're the same exact thing.
补充肯定新数据集的复杂度设计，但质疑结论在更大规模、更多 RL 和工具使用下是否仍成立。
论文的关键点似乎是：现有基准的推理复杂度相对较低，所以他们构建了新数据集 DeepRD，其推理复杂度可以任意大，并证明现有模型在问题足够复杂时会失败。复杂度定义为：把问题建模成图，并确定从源节点到目标节点所需遍历的复杂度。我的主要批评是：没有证据表明继续扩大模型规模并做更多 RL 后这个问题仍会存在。借助如今编码 agent 那样的 harness 和充分工具使用，我打赌模型能在这个推理基准上走得更远。否则，如果推理问题完全在一个上下文窗口内完成，那么足够复杂的推理问题对模型来说太难是意料之中的。
alyxya查看英文原文
The key point the paper seems to make is that existing benchmarks have relatively low complexity on reasoning complexity, so they made a new dataset DeepRD with arbitrarily large reasoning complexity and demonstrated that existing models fail at a complex enough problem. Complexity is defined from the complexity of a graph created by modeling the problem as a graph and determining the traversals needed to go from some source node to a target node. My main critique is that I don't think there's evidence that this issue would persist after continuing to scale models to be larger and doing more RL. With a harness like what coding agents do these days and with sufficient tool use, I bet models could go much further on that reasoning benchmark. Otherwise, if the reasoning problem were entirely done within a single context window, it's expected that a complex enough reasoning problem would be too difficult for the model to solve.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2025Hacker News 讨论帖218 分 · 217 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选