AI 与机器学习2017arxiv.org
深度学习的失败
Failures of Deep Learning
进阶 · 约 60 分钟读完前置知识：深度学习基础、机器学习理论、组合推理
导读
这篇论文研究深度学习的失败模式与理论局限。作者构造了看似简单、但需要组合推理的合成任务——例如随机角度的线条图像按角度是否大于 90 度标为正负例，再按样本集中正例数的奇偶性打标签——用来说明深度网络在这类任务上会失败，并尝试分析其原因。
HN 关注点：HN 读者既关注“证明深度学习不能做什么”这一理论方向，也集中吐槽论文把简单想法写得极其晦涩、数学符号堆砌。
HN 评论精选
这几条评论里：支持 1 质疑 1 补充 1
质疑论文用不必要的形式化掩盖简单概念，严重增加阅读负担。
就在昨天，HN 上还有一场大讨论：学术论文如何不必要地把简单想法复杂化，主要方式是用难以理解的数学符号替代清晰解释，以显得更正式。这篇论文正是反面典型。比如第 5 页：他们想解释一个很简单的想法——生成随机角度的线条图像，根据角度是否大于 90 度把线条标为正例或负例；然后取这些样本组成集合，再根据集合中正例数量是偶数还是奇数给集合打标签。他们用了半页多、好几段密集数学符号来解释。如果你不知道 U、:、->、~ 这些符号是什么意思，就完了，因为这些符号根本没法谷歌。尤其我只是想快速浏览想法，却不得不痛苦地反向工程。甚至“偶数/奇数”的概念也被毫无必要地重新定义并复杂化成把 +1 或 -1 相乘；我挠头几分钟才明白这有什么意义。这就像读没有注释的烂代码：即使你很熟悉这门语言、知道代码在做什么，如果解释不到位，也要花很大力气才能搞懂为什么这样写。最糟的是，没人抱怨这些，因为怕显得自己蠢。
Houshalter查看英文原文
Just yesterday there was a big discussion on here about how academic papers needlessly complicate simple ideas. Mainly by replacing nice explanations with impenetrable math notation in an attempt to seem more formal. This paper is very guilty of this. E.g. page 5. They attempt to explain a really simple idea, that they generated images of random lines at a random angle. Then labelled the lines positive or negative examples, based on whether the angle was greater than 90 degrees or not. Then they take sets of these examples. And label them based on whether they contain an even or odd number of positive examples. They take several paragraphs over half a page to explain this. Filled with dense mathematical notation. If you don't know what symbols like U, :, ->, or ~ mean, you are screwed because that's not googleable. It takes way longer to parse than it should. Especially since I just wanted to quickly skim the ideas, not painfully reverse engineer them. Hell, even the concept of even or odd, is pointlessly redefined and complicated as multiplying + or - 1's together. I was scratching my head for a few minutes just trying to figure out what the purpose of that was. It's like reading bad code without any comments. Even if you are very familiar with the language and know what the code does, it takes a lot of effort to figure out why it's that way. If it's not explained properly. The worst part is, no one ever complains about this stuff because they are afraid of looking stupid. I sure fear that by posting this very comment. I actually am familiar with the notation used in this example. I still find it unnecessary and exhausting to decode.
支持证明深度学习的局限有助于理解其能力边界并催生结构理论。
我认为最激动人心、最有意思的工作，不只来自证明系统能做什么，也来自证明系统的约束，比如哥德尔、香农、Aaronson，以及其他那些敢说“不”的人——这个传统虽然不够主流，但很可贵。更好地理解深度学习不能（很好地）做什么，是理解它能解决哪些问题的肥沃材料。我非常期待这个方向有更多工作，也期待形成某种底层结构理论。
dmreedy查看英文原文
I think some of the most exciting and interesting work comes out of proving, not just capabilities, but constraints for systems, be it Gödel, Shannon, Aaronson, or any of the others in the smaller-than-desirable tradition of those who say, "No". I think a better understanding what Deep Learning can't do (well) is fertile material for better understanding the kinds of problems it can do, and am very excited to see more work in this space, and movement towards an underlying structural theory.
补充提供作者报告与直播资源。
第一作者下周将就此工作做报告（会直播并录像），属于一个表示学习 workshop：https://simons.berkeley.edu/talks/shai-shalev-shwartz-2017-3-28
csfoo查看英文原文
The lead author will be giving a talk on this work next week (which will be live streamed and recorded) as part of a workshop on Representation Learning: https://simons.berkeley.edu/talks/shai-shalev-shwartz-2017-3-28
历次讨论
- 2017Hacker News 讨论帖210 分 · 44 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选