AI 与机器学习2017arxiv.org编辑精选
基于强化学习的神经架构搜索（Neural Architecture Search with RL）
Neural Architecture Search with Reinforcement Learning
进阶 · 约 50 分钟读完前置知识：强化学习基础、CNN/RNN 结构、策略梯度
导读
论文提出用强化学习来搜索神经网络结构：一个 RNN "控制器" 逐层采样出子网络的结构（如卷积网络或循环网络），用子网络在验证集上的精度作为奖励信号，通过策略梯度训练控制器。实验在 CIFAR-10 图像分类和 Penn Treebank 语言建模上取得与当时最优结果相当或更好的表现，但搜索代价极高，需要数百块 CPU/GPU 并行。
HN 关注点：这是最早的 Neural Architecture Search 工作之一，HN 读者既惊叹于它用机器学习替代手工设计网络结构的思路，也对它的巨大算力开销和后续 meta-learning 方向展开讨论。
后来怎么样
这篇论文是神经架构搜索（NAS）领域的奠基性工作之一，推动了 AutoML 与后续大量 NAS 研究；但其原始强化学习搜索的算力代价极高，后续工作多转向更高效的搜索策略。
HN 评论精选
这几条评论里：支持 1 补充 3
支持详细总结论文方法与其算力代价，并指出它回应了“深度学习只是手工拼组件”的批评。
极简论文速览——人类通常手工设计神经网络的组件以及组件之间如何连接的图结构。这项工作构建了一个“控制器”神经网络，用强化学习生成许多神经网络中的两个核心组件：RNN 和 CNN。这个过程密集而缓慢，RNN 和 CNN 分别需要 400 个 CPU 和 800 个 GPU，但在语言建模和视觉分类上分别达到优于或接近当时最优的水平。该论文正在 ICLR 2017 审稿中，是我最兴奋的论文之一。我之前写过一篇文章《在深度学习中，架构工程是新的特征工程》，讨论了一个常被忽视的事实：现代深度学习论文中的大量工作只是以不同组合方式拼装组件。这是我认为第一个为我的抱怨/担忧提供可行答案的工作。论文解决两个问题：一是架构优化通常是人类指导不佳的黑魔法，二是人类很少花时间为特定任务定制，而是追求通用性。Zoph 和 Le 通过大量实验，让一个神经网络为另一个神经网络生成架构，并在视觉（分类）和文本（语言建模）上替代了 CNN 和 RNN 组件……
Smerity查看英文原文
Extreme paper tldr - Humans usually construct neural network components and the graph of how they fit together by hand. This work sets up a "controller" neural network that constructs two core components in many neural networks, an RNN and a CNN, through reinforcement learning. This is an intensive and slow process, requiring 400 CPUs and 800 GPUs for the RNN and CNN respectively, but achieves better than or near state of the art results for language modeling and vision classification respectively. This paper is currently under review for ICLR 2017 and is one of the papers I was most excited about. I previously wrote an article, "In deep learning, architecture engineering is the new feature engineering"[1], which discusses the (often ignored) fact that much of the work in modern deep learning papers is just assembling components in different combinations. This is one of the first works that I feel provides a viable answer to my complaint/concern. The paper itself tackles two problems - first, that optimizing architecture is usually black magic directed poorly by humans, and second, humans rarely spend their time tailoring towards a specific task, instead seeking generality. Zoph and Le do this by having one neural network generate the architecture for a later one through a large series of experiments. They perform experiments in both vision (classification) and text (language modeling), replacing the convolutional neural network component and the recurrent neural network component respectively. First is that many of the choices regarding constructing the neural network architecture are somewhat arbitrary and only hit upon experimentally by the practitioners themselves. Andrej Karpathy noted in one of his lectures (paraphrased) "Start with an architecture that works, then modify from there" - mainly as there's a lot of "black magic" in these architectures that has only been discovered by spilling blood to the experimental god of a hundred GPUs and/or "graduate student descent" (i.e. where you lock a poor grad in a room for an indeterminate period of time and tell them to do better on task X). Being able to turn to a neural network to run this painful search for you instead is a good idea - assuming you have the large number of GPUs or CPUs necessary. In the paper they use 400 CPUs for the language modeling search and 800 GPUs for the CNN classification search! The other is whether we should generalize or specialize these architectures. There are many variants of architectures that are not built for or tested against each possible new task. For example, within recurrent neural networks (RNNs) we have the RNN/GRU/LSTM/QRNN/RHN/... and a million minor minor variants between them, each of which perform slightly different depending on the task. While we would like to imagine the architectures that humans make would get progressively closer to "the perfect generic RNN cell" over time, it makes sense that certain cells are, could, or should be optimized to a specific task. Seeking generality isn't always the correct answer. Humans want to seek generality as we don't have the time to tailor to each specific task - but what if we could? Maybe in that situation Occam's razor is actually an impediment to our thinking. While this is early days, and hugely resource intensive, it is likely to get more feasible over time either as we get more computing power or become smarter regarding how we use it. As a researcher in neural networks, I don't consider this a threat, but instead a useful tool, likely in the same way that a compiler likely only helped assembly programmers. If people are interested, I can write an article covering many of the details of this paper like I did for Google's Neural Machine Translation architecture[2]. In that article I try to step through how these systems work from the ground up, and the reasoning behind many of the decisions in the paper, hopefully in an understandable manner for a general audience. P.S. M…
补充补充背景：一作来自 Google Brain Residency 项目，并称赞该项目产出高效。
第一作者是 Google Brain Resident 项目的成员（附链接）。Brain Residency 是开启深度学习与机器学习研究生涯的极佳项目，这些新研究者能如此迅速地产出这样出色的工作，令我印象深刻。披露：我在 Google Brain 工作。
ericjang查看英文原文
The primary author is a Google Brain Resident (https://research.google.com/teams/brain/residency/). The Brain Residency is a really great program for starting a career in Deep Learning and ML research, and I'm really impressed by how quickly these new researchers churn out great work like this. disclosure: I work at Google Brain
补充建议做 meta-meta-learning，并把网络规模纳入搜索目标以压缩模型。
我很想看到一个“用 Neural Architecture Search 来生成 Neural Architecture Search”的未来研究项目。元元学习。我喜欢用网络去改进生成其他网络的网络这个想法。另外，网络规模也可以作为评估的一部分，从而在最小化网络的同时最大化精度。
jorgemf查看英文原文
I would love to see a Neural Architecture Search that create Neural Architecture Search as a future research project. Meta-meta-learning. I like the idea of improving the network which creates other networks. Also the size of the network can be used as part of the evaluation in order to minimize the networks and maximize the accuracy.
补充推测模块化串联多个网络是实现通用智能的路径。
我认为把神经网络串联起来，是它们获得某种通用性的方式。假设你有一个机器人，要从台面上拿起一罐啤酒。你说“拿起那罐啤酒”并指过去。第一个神经网络解析语音和视觉输入；第二个神经网络根据第一个网络解读出的信息，选择合适的后续神经网络来继续任务——它挑一个负责行走，一个负责抓取。
gallerdude查看英文原文
I think this is the way that Neural Networks achieve some modicum of generality - chaining them together. Let's say you have a robot that you want to grab a can of beer off the counter. You say "grab that beer" and point to it. The first neural network interprets the speech and visual input. A second neural network chooses the proper neural nets to continue the task based on the information interpreted from the first net - it picks one for walking and one for grabbing.
历次讨论
- 2017Hacker News 讨论帖116 分 · 38 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选