AI 与机器学习2014arxiv.org编辑精选
深度神经网络很容易被愚弄
Deep Neural Networks Are Easily Fooled
进阶 · 约 45 分钟读完前置知识：神经网络基础、图像分类、进化算法
导读
这篇论文研究深度神经网络为何会被人类看来难以识别或几乎无意义的图像欺骗，并展示可以自动生成让网络以高置信度误分类的样本。核心结论是这类模型对特定输入模式或扰动很脆弱，高置信度输出并不等于真正理解图像。
HN 关注点：是否暴露了神经网络/AI 的根本缺陷，讨论焦点包括对抗样本是否普遍、与人类视错觉和生物伪装的类比，以及如何缓解。
后来怎么样
这篇论文让“高置信度误分类”和对抗脆弱性进入更广泛视野，与 Szegedy 等工作共同推动对抗样本与鲁棒性研究成为深度学习的重要方向，并催生大量攻击与防御后续工作。
HN 评论精选
这几条评论里：质疑 1 补充 3
质疑对抗样本并非神经网络独有，应避免借标题夸大结论。
这些论点今年早些时候已由 Szegedy 等人在论文中提出，Hinton 上个月也在 Reddit AMA 中讨论过。该结果并非神经网络独有，类似技术也能骗过逻辑回归。问题在于，训练好的网络最终严重依赖某些激活路径；在完全了解网络的情况下，可以精确针对这些路径，使网络把人类看来几乎察觉不到变化的样本误分类。理解对抗样本很重要，但借耸动标题对所有神经网络乃至一般智能或整个 AI 研究作出以偏概全的断言是不合理的。
zackchase查看英文原文
These arguments were introduced by Szegedy et. al. earlier this year in this paper: http://cs.nyu.edu/~zaremba/docs/understanding.pdf. Geoff Hinton addressed this matter in his Reddit AMA last month. The results are not specific to neural networks (similar techniques could be used to fool logistic regression). The problem is that ultimately a trained network relies heavily on certain activation pathways which can be precisely targeted (given full knowledge of the network) to fool networks into misclassification on data points which might to a human seem imperceptibly changed from those which are correctly classified. It is important to understand adversarial cases, but unreasonable to get carried away with sweeping pronouncements about what this does or doesn't about all neural networks, let alone intelligence generally, or the entire enterprise of AI research, as seems to happen after a splashy headline.
补充易被骗可能是模式匹配系统的通病，关键在上下文训练或逆向失败模式。
人类擅长模式匹配，也擅长发现并不存在的模式，不仅视觉如此，对概率、随机性甚至因果也如此。所幸大脑很复杂，能根据上下文识别出灰泥墙或云，否定对面孔、独角兽等的误识别。因此，我们模仿智能的尝试容易被骗并不奇怪？无数物种正是利用捕食者的模式匹配错误，把自己伪装成树叶、树枝或捕食者避之不及的有毒动物。DNN 还年轻，路还长，所以这是理论根基上的根本问题，还是只需用更多上下文数据训练？如果我们能像自然选择中的神一样窥视捕食者（算法）的大脑并逆向工程其失败（猎物伪装），是否有可能得到准确的 DNN？
akiselev查看英文原文
We humans are as brilliant at pattern matching as we are in finding patterns that aren't really there, not just with our vision but with our understanding of probability , randomness, and even cause & effect. Thankfully, our brains are very complicated machines that can recognize a stucko wall or a cloud and invalidate the false identification of a face or unicorn or whatever based on that context. With that in mind, is it really surprising that [m]any of our attempts at emulating intelligence can be easily fooled? An untold number of species have evolved to do exactly the same thing: exploit the pattern matching errors of predators to disguise themselves as leaves or tree branches or venomous animals that the predator avoids like the plague. DNNs seem to be relatively new and we've got a long ways to go, so is this a fundamental problem with the theoretical underpinning or do we just need to train them with far more contextualized data (for lack of a better phrase)? Is there any chance of us having accurate DNNs if we can, as if gods during the course of natural selection, peek into the brain of predators (algorithms) and reverse engineer failures (disguises for prey) like this?
补充与人类视错觉类似，可能是类神经网络系统的普遍性质。
没读全文我不敢多说……但这和人类会中的各种视错觉有多大区别？我们自己都分不清兔鸭图，所以这是否只是所有类神经网络系统的普遍性质：总会有大片未经特定选择压力训练的错误分类区域？
monochr查看英文原文
I have nothing intelligent to say without reading the full paper... ...But, how different is this from the various optical illusions humans fall for? I mean we can't exactly tell the difference between a rabbit and duck ourselves[1] so isn't it just a universal property of all neural-network like systems that there will be huge areas of mis-classifications for which there hasn't been specific selection? [1] http://mathworld.wolfram.com/Rabbit-DuckIllusion.html
补充可用多视角或平移交叉验证缓解，人脑靠扫视多次采样。
该漏洞利用的是 NN 权重的不完美。要避免这种错配，只需把同一图像平移 1 像素（假设按像素识别），再交叉核对结果看是否出错。人脑识别更好，是因为能从许多略微不同的角度多次采样图像；扫视存在是有原因的。
cLeEOGPw查看英文原文
The vulnerability exploits imperfections in the NN weights. To avoid this kind of mismatch all you need to do is shift the same image by 1 pixel (assuming recognition is done per pixel), and you can cross check results to check if an error occurred. Human brain recognizes better because it can sample the image many times from many slightly different angles. There's a reason saccade (http://en.wikipedia.org/wiki/Saccade) exists.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2014Hacker News 讨论帖183 分 · 60 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选