AI 与机器学习2018arxiv.org
当算法令我们惊讶时
When algorithms surprise us
入门 · 约 45 分钟读完前置知识：进化算法基础、机器学习基础
导读
这篇论文（arXiv:1803.03453）收集整理了进化算法和机器学习系统在优化过程中产生“意外”聪明或荒诞解法的诸多案例，探讨算法为何会以设计者未预料到的方式达成目标。文章还从哲学角度讨论了“惊讶”的不同类型，以及如何判断算法行为是否真正出乎专家意料。
HN 关注点：HN 读者长期对“奖励函数被钻空子”的案例津津乐道，这篇论文系统性地汇总了这类趣闻，引发大家分享自己写适应度函数被算法“耍”的亲身经历。
HN 评论精选
这几条评论里：质疑 1 补充 1 亲历 2
亲历亲历：进化算法很容易钻适应度函数的空子，设计好的适应度函数极难
大约20年前，我满怀期待地启动了演化了几周的机器人算法。我早已意识到必须调整适应度函数，给“击杀”加分而不只是“存活”——因为第一版算法造出的机器人不是像我设想的那样逐个精准消灭对手，而是直接冲向最近的角落躲起来等死。第二次尝试时机器人还是躲进角落，但这次会随机开火，偶尔碰运气击杀对手。演化算法就此陷入瓶颈，加上我不想让电脑连续几天只跑机器人对战模拟，就放弃了。但我学到两点：演化大概率是真的存在；写适应度函数真的非常非常难。
kpil查看英文原文
I think it was almost 20 years ago when I expectantly started the robot algorithm that had slowly been evolving for weeks. I had already realized that I had to adjust the fitness function to give points for killing, not just surviving, since the first attempt had created robots that instead of skillfully picking off the opponents one by one - as I had envisioned - took speed for the nearest corner and then just sat there waiting to be killed by one of the other traditionally programmed robots. I realized that the corner strategy made sense as they got the most protection there, but it was a bit unsatisfactory to watch... In the second try the robots scurried to their usual corners, waiting to be killed, but this time firing randomly, occasionally killing one of their opponents by luck before they got hit. The evolutionary algorithm sort of hit a plateau there, and besides I kind of wanted to use my computer for other things rather than spending days and days simulating robot fights. But I learned two things: * Evolution most likely is a thing. * It's really really hard to write fitness functions...
质疑质疑：很多“意外”结果对专家并不意外，有机形态本就是演化优化的近似最优解
重要的是要确认这些算法产出的结果对领域专家来说是否真的“意外”。我记得有个博客作者惊讶于一个优化物理结构（最少材料、最大强度）的程序产出了类似有机生物的形态。这对我毫不意外——有机结构经过数亿年演化，在特定任务上已接近最优，如果优化软件没产出有机形态才奇怪。而且该软件也并不比真实有机结构更好，因为软件假设了各向同性和均质性，而有机结构并非如此。
btrettel查看英文原文
It's important to make sure that what these algorithms actually produce is surprising to subject matter experts. I can recall, but can not find at the moment, a tumblr post where the blogger was surprised that a computer program which optimized a physical structure to minimize material while maximizing strength ended up looking organic. This didn't surprise me at all. Organic structures evolved over millions or billions of years and probably are nearly optimal at accomplishing a particular task. I'd be surprised if the optimization software didn't produce something that looked organic. It's not like the optimization software was better than actual organic structures either, which are neither isotropic (in organic structures the strength varies depending on the direction) or homogeneous (in organic structures the strength varies depending on the location) as the software had assumed.
补充作者亲自到场分享论文相关的动图和媒体报道
大家好，希望你们喜欢这篇论文。这里有两个 AI 想出的巧妙解法的有趣动图（Twitter 链接），另外 Popular Mechanics 和 New Scientist 上有更简短的趣闻摘要报道。
jeffclune查看英文原文
Hello all. We hope you enjoy the paper. Here are two fun gifs of clever solutions AI came up with: https://twitter.com/jeffclune/status/974718199722795008 https://twitter.com/jeffclune/status/973605950266331138 Here are some press articles which provide a shorter summary of some fun anecdotes. popularmechanics.com/technology/robots/a19445627/the-hilarious-and-terrifying-ways-algorithms-have-outsmarted-their-creators/ https://www.newscientist.com/article/8-8-hilarious-ways-ai-has-outsmarted-us-to-get-the-job-done/ (paywalled unfortunately)
亲历亲历：遗传算法钻空子发现“零交易”是最优解，说明适应度函数常偏离真实意图
用进化压力引导软件时，要记住你声明的目标（通过适应度函数表达）往往并不反映你真正的意图。比如我用遗传算法选股票交易时，试图最大化总净收益/交易次数，GA 很快发现最优答案是 0 次交易。事后想想：废话！可我是想做交易的啊。“唯一的制胜之道就是不玩”。
mring33621查看英文原文
When using evolutionary pressures to guide software, you have to remember that your stated goal (via fitness functions) often does not truly reflect your intended goal. For example, when I was using genetic algorithms to pick stock trades, I tried to maximize total_net_gain_loss / number_of_trades. The GA quickly figured out that 0 trades was the best answer. In hindsight, Duh! But I wanted to make trades! "The only winning move is not to play"
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2018Hacker News 讨论帖86 分 · 19 评
- 2018Hacker News 讨论帖112 分 · 18 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选