AI 与机器学习2024arxiv.org
LLMs 将始终产生幻觉，我们需要接受这一点
LLMs Will Always Hallucinate, and We Need to Live with This
入门 · 约 30 分钟读完前置知识：概率论基础、大语言模型基本原理
导读
这篇论文从数学上论证大语言模型的“幻觉”具有必然性，无法被完全消除，因此主张我们应学会与幻觉共存，而不是追求彻底消除。作者的核心论点是：生成虚假信息是概率性文本生成过程的固有结果，而非可修复的偶发故障。
HN 关注点：“幻觉”究竟是不是模型故障，以及理论上的非零概率对实际风险缓解意味着什么。
HN 评论精选
这几条评论里：质疑 1 补充 2 亲历 1
补充“幻觉”一词误导人们把概率生成的正常产物当成可修复的故障；缓解幻觉与对齐本质上是同一问题。
有数学证明固然好，但老实说，如果我们给“在生成概率性文本过程中产生虚假信息”这个概念换个名字，整个误解本来可以避免。“幻觉”听起来像是模型正常运作中出了岔子，暗示只要找出问题就能消除它并恢复LLM的正常认知功能。但问题在于，模型的正常运作本就是生成听起来合理的文本。“幻觉”不是模型故障，而是我们对输出文本的价值判断；它只说明生成的文本不适合用途。这样看，减轻幻觉和实现“对齐”其实是同一个问题，二者无法分开解决。
lolinder查看英文原文
> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is going awry in the normal functioning of the model, which subtly suggests that if we could just identify what went awry we could get rid of the problem and restore normal cognitive function to the LLM. The trouble is that the normal functioning of the model is simply to produce plausible-sounding text. A "hallucination" is not a malfunction of the model, it's a value judgement we assign to the resulting text. All it says is that the text produced is not fit for purpose. Seen through that lens it's obvious that mitigating hallucinations and creating "alignment" are actually identical problems, and we won't solve one without the other.
补充幻觉类似人类未经反思的“第一反应”，应给LLM反思空间，不能以即时回答评判智能。
幻觉难道不就是模型把对问题的第一个可能答案直接说出口的结果吗？人类不会这样。首先，大多数问题我们以前被问过，也犯过错并记住了，所以不会重复；其次，我们（至少有些人）会先想后说，把最初反应与其他已知信息联系起来，常不自觉地做“合理性检查”。因此，不应期望LLM不经过反思就立刻给出正确答案。事实上，观察自己的思考会发现，思维常扮演不同角色或人格，答案多是内部讨论与妥协的结果；我们还会想象说出某答案的后果，而这只有在脑中形成初稿后才可能。所以仅凭LLM对提示的第一“直觉反应”来评判其智能，很可能是错误的。
leobg查看英文原文
Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing it, we relate that thought to other things we know. We may do “sanity checks“ internally, often habitually without even realizing it. Therefore, we should not expect an LLM to generate the correct answer immediately without giving it space for reflection. In fact, if you observe your thinking, you might notice that your thought process often takes on different roles and personas. Rarely do you answer a question from just one persona. Instead, most of your answers are the result of internal discussion and compromise. We also create additional context, such as imagining the consequences of saying the answer we have in mind. Thoughts like that are only possible once an initial “draft” answer is formed in your head. So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is probably misguided. Let me know if you need any further revisions!
亲历以对待易犯错人类的方式使用LLM，接受风险但不盲信，实际体验良好。
我把LLM当作会犯错的实体，就像对待人类一样。我不会无条件信任其输出，在接受它帮忙完成任务时也知道自己在承担一定风险。带着这种心态，我使用GPT-4o/ChatGPT和GitHub Copilot的体验总体非常正面，我每天都会频繁使用它们。
bicx查看英文原文
I treat LLMs like a fallible being, the same way I treat humans. I don’t just trust output implicitly, and I accept help with tasks knowing I am taking a certain degree of risk. Mostly, my experience has been very positive with GPT-4o / ChatGPT and GitHub copilot with that in mind. I use each constantly throughout the day.
质疑非零概率的理论证明不等于无法实际缓解风险；关键看能否指数级降低发生概率。
好吧——幻觉概率永远非零。宏观物体发生量子隧穿的概率也非零，但没人因此主张我们“必须接受这一点”。证明某事件概率不可能降到0%的理论固然不错，但在实践中，它几乎说明不了我们能否把该概率指数级降低，从而有效缓解风险。
mxwsn查看英文原文
OK - there's always a nonzero chance of hallucination. There's also a non-zero chance that macroscale objects can do quantum tunnelling, but no one is arguing that we "need to live with this" fact. A theoretical proof of the impossibility of reaching 0% probability of some event is nice, but in practice it says little about whether we can exponentially decrease the probability of it happening or not to effectively mitigate risk.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2024Hacker News 讨论帖291 分 · 261 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选