AI 与机器学习2025arxiv.org
超越语义：无推理中间 token 的不可思议有效性
Beyond Semantics: Unreasonable Effectiveness of Reasonless Intermediate Tokens
进阶 · 约 45 分钟读完前置知识：思维链（CoT）基础、Transformer 基础
导读
这篇论文研究思维链（CoT）中间 token 是否真的承载语义。作者将训练数据中的推理轨迹替换为与问题无关的噪声或损坏内容，发现模型性能基本不变，某些情况下甚至更好，说明中间 token 的作用可能不在于其推理内容，而在于提供了额外的计算空间。
HN 关注点：这一结论对 CoT 机制的挑战：如果推理内容无关紧要，CoT 的价值可能只是让模型获得更多计算，而非真正的推理。
HN 评论精选
这几条评论里：支持 1 质疑 2 补充 1
支持CoT 的价值可能只是额外计算，而非有意义的推理
有趣的是，用与问题无关的噪声轨迹训练，性能基本不变甚至更好。我们大概都见过模型思考过程跑偏但最后给出正确答案的情况。这暗示 CoT 有用的原因是模型获得了更多内部计算，而不是产出的文字有意义。奇怪的是至今没人提出好的按 token 自适应计算方案，也许可以完全跳过 CoT。
modeless查看英文原文
> we then train models on noisy, corrupted traces which have no relation to the specific problem each is paired with, and find that not only does performance remain largely consistent with models trained on correct data, but in some cases can improve upon it This is the interesting part. We've probably all had the experience where the model is going off the rails during the thinking process but somehow spits out the right answer at the end. Apparently the reasoning doesn't even need to be correct during training? I guess it suggests to me that the reason CoT helps is that the model gets more compute to think internally, not that the words it produces are meaningful. I'm surprised nobody has come up with a good scheme for adaptive compute per token yet. Maybe we can skip CoT entirely.
质疑推理发生在潜空间而非 token 空间，错误 token 不代表没有有效推理
语言模型产出的是下一个 token 的分布，采样出的序列包含错误推理，不代表潜空间中没有有用的推理轨迹。认为 transformer 在 token 空间推理是误解：token 之间不相互注意，高维潜变量之间才相互注意，解码器最后一层可以访问之前所有潜变量的完整空间。
valine查看英文原文
I think it’s helpful to remember that language models are not producing tokens, they are producing a distribution of possible next tokens. Just because your sampler picks a sequence of tokens that contain incorrect reasoning doesn't mean a useful reasoning trace isn’t also contained within the latent space. It’s a misconception that transformers reason in token space. Tokens don’t attend to other tokens. High dimensional latents attend to other high dimensional latents. The final layer of a decoder only transformer has full access to entire latent space of all previous latents, the same latents you can project into a distribution of next tokens.
补充噪声 token 中可能存在人类不可读但与答案相关的隐含信号
即使训练在人类语言上，token 也可以带有只有 AI 能读懂的"潜台词"，而且 AI 自己也无法向我们解释。就像英语中通过措辞暗示话题与政治或性相关。推理训练会放大这种潜台词，因为任何碰巧与正确结果相关的随机噪声都会被强化。也许可以训练两个不同模型并交换推理轨迹来抑制，但不同模型可能因训练数据相关性产生相似潜台词，tokenization 差异大的模型或许更不容易。
nullc查看英文原文
Even when you train AI on human language, the tokens can have "subtext" that is only legible to the AI. And, unfortunately, it's not even legible to the AI in ways that it could ever explain it to us. It's no different than how in English we can signal that a statement is related to a kind of politics or that it's about sex through particular word and phrase choice. Training for reasoning should be expected to amplify the subtext, since any random noise in the selection that by chance is correlated with the right results will get amplified. Perhaps you could try to dampen this by training two distinct models for a while, then swap their reasoning for a while before going back-- but sadly distinct models may still end up with similar subtexts due to correlations in their training data. Maybe ones with very distinct tokenization would be less likely to do so.
质疑批评标题滥用 Wigner 经典论文的命名模式
"Unreasonable Effectiveness of ..."这种标题模式被用得太滥了。原论文（数学在自然科学中不合理的有效性）确实涉及深刻的哲学谜题，但现在什么论文都往上套。
ngruhn查看英文原文
Man that "Unreasonable Effectiveness of ..." pattern is getting a bit overused. With the original paper [1] you could still say that there really is some deeply philosophical mystery. But they now slap that on everything. [1] https://en.m.wikipedia.org/wiki/The_Unreasonable_Effectiveness_of_Mathematics_in_the_Natural_Sciences
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2025Hacker News 讨论帖138 分 · 66 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选