AI 与机器学习2018arxiv.org
用六个神经元玩 Atari
Playing Atari with Six Neurons
进阶 · 约 60 分钟读完前置知识：强化学习基础、神经进化、自编码器
导读
这篇论文提出用极小的神经网络玩 Atari 游戏：先用压缩器（如自编码器/向量量化）将游戏画面降维，再仅对低维控制器应用神经进化算法。作者称控制器只需约六个神经元即可取得可观成绩，且实验在单台工作站上即可完成，无需大规模算力。
HN 关注点：展示了用极少量参数和普通硬件就能做强化学习，与 AlphaGo 式大规模算力路线形成鲜明对比，讨论焦点是神经进化与无监督预处理的组合思路。
HN 评论精选
这几条评论里：支持 1 补充 3
支持从业者认为小参数量+单机可跑是巨大优势，并指出这与 World Models 的分层设计思路一脉相承
作为正在做强化学习/神经进化的人，我觉得这非常令人兴奋。参数更少总是更好——论文实验在单台工作站上运行，而非 AlphaGo 那样的大规模 TPU 集群，意味着迭代更快、普通研究者更容易上手。这种“压缩器+控制器”的分层设计（只对低维控制器做神经进化）让我想起 Ha 和 Schmidhuber 的 World Models 论文：VAE 约 440 万参数、RNN 约 170 万参数，而最终控制器只有 1088 个参数！虽然最近有工作表明神经进化可以扩展到数百万参数，但“进化尽可能少的参数、辅以自编码器或向量量化”的思路正在兴起。
pjrule查看英文原文
As someone working on a reinforcement learning/neuroevolution problem right now, I find this to be extremely exciting. Fewer parameters, ceteris paribus, is always better—the fact that the experiments in this paper were run on one workstation, rather than on a massive farm of TPUs à la AlphaGo, implies quicker development iteration time and more accessibility to the average researcher. The staging of components in this paper (compressor/controller), where neuroevolution is only applied to a low-dimensional controller, reminds me of Ha and Schmidhuber's recent paper on world models (which is briefly cited) [1]. They employ a variational autoencoder with ~4.4M parameters, an RNN with ~1.7M parameters, and a final controller with just 1,088 parameters! Though it's recently been shown that neuroevolution can scale to millions of parameters [2], the technique of applying evolution to as few parameters as possible and supplementing with either autoencoders or vector quantization seems to be gaining traction. I hope to apply some of the ideas in this paper to multiple co-evolving agents... [1]. https://worldmodels.github.io [2]. https://arxiv.org/abs/1712.06567
补充感叹该领域仍有大量未被探索的简单组合方向
很酷的文章，信息量很大。有一点引起我的注意：“据我们所知，此前唯一将无监督学习作为神经进化预处理器的工作是（引用）。”这个领域居然还有这么多唾手可得的低垂果实，令人惊叹。
kthejoker2查看英文原文
Cool article, lots to digest, one thing caught my eye: "To the best of our knowledge, the only prior work using unsupervised learning as a pre-processor for neuroevolution is (cite)." Just amazing how much low-hanging fruit there still is in the space.
补充自学者分享 RL 学习资源和术语混乱带来的困惑
我最近在猛啃强化学习的文章和视频。RL 的整体概念（智能体、动作、状态等）容易理解，但一些细节和流程相当令人困惑。我倾向于归咎于术语和实现的不一致，但怀疑更多是因为我以开发者而非研究者的视角切入——只看代码而不完全理解机制背后的科学。推荐几个资源：reinforce.io 及其 tensorforce 库、Karpathy 的 RL 博文等。补充一点：新内容发布的速度超过了我能吸收的速度，总感觉在追赶。
markatkinson查看英文原文
I have been wolfing down RL articles, videos and publications after a intro to deep learning via Manning's Deep Learning for some time now and while the overall concept of RL is easy to grasp (agents, actions and state etc) some of the finer details and processes are quite confusing. I am tempted to blame inconsistency across terminology and implementations for this lack of understanding but I suspect it has more to do with approaching this field through the lens of a developer and not a researcher or academic. Trying to understand the code without grasping the "science" of the mechanisms completely. Either way if you feel to be in a similar spot check out this resource: https://reinforce.io and their respective Github repo: https://github.com/reinforceio/tensorforce. Just reading through their code, and documentation has made a lot of the concepts clearer. And a few more resources I found really helpful: http://karpathy.github.io/2016/05/31/rl/ https://www.analyticsvidhya.com/blog/2017/01/introduction-to-reinforcement-learning-implementation/ https://www.oreilly.com/ideas/reinforcement-learning-with-tensorflow Edit: My point that I forgot to mention was that I always feel like I am playing catch-up to understand what is going on half the time as the amount of new content being released exceeds what I can absorb.
补充补充论文的开源代码链接
这是对应的 GitHub 代码库：https://github.com/giuse/DNE/tree/nips2018
kthejoker2查看英文原文
And the Github library: https://github.com/giuse/DNE/tree/nips2018
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2018Hacker News 讨论帖169 分 · 30 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选