AI 与机器学习2025arxiv.org
不需要博士知识：大型语言模型的推理挑战
PhD Knowledge Not Required: A Reasoning Challenge for Large Language Models
入门 · 约 30 分钟读完前置知识：大语言模型基础、tokenization 基本概念
导读
这篇论文提出一个面向大语言模型的推理挑战/基准，试图在不要求“博士级知识”的情况下考察模型推理能力；从讨论看，题目涉及字符计数、字母替换或文字谜题类问题。讨论中有人观察到，模型的推理轨迹里有时已经出现正确答案，但模型最终没有采纳。
HN 关注点：HN 读者一方面质疑“PhD Knowledge”这一命名，另一方面争论这类字符操纵题到底是在测推理，还是 tokenization 造成的 gotcha。
HN 评论精选
这几条评论里：质疑 3 补充 1
质疑批评“PhD Knowledge”把博士误读为知识/智商标签，而非研究贡献。
“PhD Knowledge”这个说法让我很不适。博士的核心是对领域作出贡献，而不是标准化考试分数，也不是某种替代 IQ 的奇怪身份标签。
mkoubaa查看英文原文
The term "PhD Knowledge" makes me cringe. PhD is about contribution to a field, not a standardized test score or a weird status-bearing substitute for IQ.
补充指出模型推理轨迹常已包含正确答案，却未能最终识别和采纳。
有意思的是，这些模型的推理过程里常常已经出现了正确答案，但模型自己没意识到。比如 Problem 3（“Dry Eye”），R1 已经想到“Dry”和“eye”都是三个字母，也检查了是否押韵；Problem 8（“Foot nose”）里它也想到“foot”+“note”，但没意识到“note”不是身体部位。
rahidz查看英文原文
What is so interesting to me is that the reasoning traces for these often have the correct answer, but the model fails to realize it. Problem 3 ("Dry Eye"), R1: "Wait, maybe "cubitus valgus" – no, too long. Wait, three letters each. Let me think again. Maybe "hay fever" is two words but not three letters each. Maybe "dry eye"? "Dry" and "eye" – both three letters. "Dry eye" is a condition. Do they rhyme? "Dry" (d-rye) and "eye" (i) – no, they don't rhyme. "Eye" is pronounced like "i", while "dry" is "d-rye". Not the same ending." Problem 8 ("Foot nose"), R1: "Wait, if the seventh letter is changed to next letter, maybe the original word is "footnot" (but that's not a word). Alternatively, maybe "foot" + "note", but "note" isn't a body part."
质疑批评这类题是 tokenization 不适配的字符操纵 gotcha，不能代表真正想测的推理能力。
我不喜欢这类“陷阱题”，它们测的不是我们真正关心的能力。就像数 strawberry 里有几个 R，很多题是字符计数或字符操纵，而 tokenization 并不擅长这些。工程师当然能想出办法专门训练，但那像是在优化错误的目标；在我看来，字符排列是“Software 1.0”的问题，不该交给 LLM，就像不会让 LLM 去算两个大数相乘，而会用计算器。
windsignaling查看英文原文
I'm not a fan of these "gotchas" because they don't test for what we really care about. Like counting the number of R's in strawberry, many of these are character-counting or character manipulation problems which tokenization is not well-suited for. I'm sure an engineer could come up with a clever way to train for this, but that seems like optimizing for the wrong thing. IMO these questions go in the wrong direction. Character permutation is a problem for "Software 1.0", not LLMs. Just as you wouldn't use an LLM to multiply 2 large numbers, you'd use a calculator.
质疑认为把博士等同于死记硬背是反智，博士更核心的是科研训练。
好像反智浪潮还不够，现在“博士”又成了一个暗示最高水平死记硬背者的类别。我猜这种局面很难扭转，但博士远比积累知识更关乎作为科学家的学徒训练和真实世界训练。抱歉这是一段发泄，我只是受够了这种愚蠢。
lokimedes查看英文原文
As if the whole anti-intellectual hunt wasn’t enough, now PhD is a category implying holder of rote-knowledge at the highest level. I guess it is hopeless to fight this, but a PhD is 100x more about the apprenticeship and real-world training as a scientist than any accumulated knowledge beyond ones prior training. I know this is a rant, sorry, just so tired of the stupidity.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2025Hacker News 讨论帖174 分 · 80 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选