AI 与机器学习2024nature.com编辑精选
AI 模型在递归生成数据上训练时发生崩溃
AI models collapse when trained on recursively generated data
进阶 · 约 45 分钟读完前置知识：机器学习基础、生成模型/LLM、概率分布与采样
导读
这篇 Nature 论文研究在递归使用模型生成数据训练时出现的“模型坍塌”问题：不加筛选地把模型输出当作训练数据会导致模型不可逆的缺陷，分布逐渐退化。论文强调关键在于“不加区分地”使用合成数据。
HN 关注点：这一结论是否适用于当前大厂实践——各大 AI 实验室早已在训练中使用合成数据，讨论焦点是“不加筛选”与“有控制地使用”的区别。
HN 评论精选
这几条评论里：质疑 4
质疑论文结论只在“不加筛选”使用合成数据时成立，大厂已有控制手段，问题被夸大了
关键词是“不加区分地”。所有大 AI 实验室至少一年来都在用合成数据训练，但他们是刻意为之的。我认为“模型坍塌”问题如今并不特别重要，训练模型的人似乎已经把它控制得很好。
simonw查看英文原文
> We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models The key word there is "indiscriminate". All of the big AI labs have been training on synthetic data for at least a year at this point, but they're doing so deliberately. I don't think the "model collapse" problem is particularly important these days. The people training models seem to have that well under control.
质疑质疑：既然无法区分人类与 LLM 文本，爬取网络数据训练不可避免会混入模型输出，这问题怎么解决？
我一定是漏掉了什么。把系统输出当作已验证的输入来训练，显然是禁忌。我说的不是刻意使用合成数据，而是把网上找到的一切都当作“真实”的人类生成文本。多个 LLM 的输出会进入网络，且目前没有任何指标或算法能可靠区分人类文本和 LLM 文本。我到底漏掉了什么？
mcswell查看英文原文
I must be missing something. Training on the output of your system as if it were validated input seems like an obvious no-no. I'm not talking about using synthetic data (however that might be created in this situation), but rather using anything and everything found on the web as if it were "real", i.e. as if it were human-generated texts rather than the output of the LLM. In this case of course there are multiple LLMs that are creating text which finds its way to the web, but to the extent that the output of the different LLMs have commonalities, this still seems problematic. And afaik, there are no metrics or algorithms that reliably distinguish between human-generated and LLM-generated text, at least not for the current generations of LLMs. What am I missing?
质疑举出反例：OpenAI 和 Anthropic 用合成数据训练反而提升了模型
与此同时，OpenAI、Anthropic 都在用 AI 生成的数据训练并改进模型，而且有效。
Kuinox查看英文原文
Meanwhile OpenAI, Anthropics, trains on AI generated data to improve their models, and it works. https://openai.com/index/prover-verifier-games-improve-legibility/ https://www.anthropic.com/research/claude-character
质疑类比：教科书理论（如模型坍塌）与真实世界实践常常不符，实际从业者早已找到应对之道
我读经济学时学过最后通牒博弈：标准理论预测接受者会接受任何大于零的出价，但现实中低于约 30 美元的出价通常被拒绝，因为对方是会感到被亏待的真实的人。唯一接受 1 美元出价的，恰恰是学过经济理论的人。“公地悲剧”也是类似——现实中世界各地都有社区通过实践和信念体系成为共享资源的负责任管理者，而非理论预测的崩溃。
roughly查看英文原文
Back when I was getting my econ degree, we were taught about the Ultimatum game, which goes like this: You get two participants who don't know each other and will (ostensibly) never see each other again. You give one of them $100, and they make an offer of some portion of it to the other. If the other accepts, both parties keep their portion - so, if A offers B $20, and B accepts, A keeps $80 and B keeps $20, if B rejects, both parties get nothing. Standard economic theory suggests A can offer $1 and B will accept, because otherwise B gets nothing. Spoiler for those of you who haven't seen how standard economic theory plays out in real life, that's not how the game went - typically, offers below ~$30 or so got rejected, because B was a real feeling person who felt like they were getting screwed and opted to punish A for doing so. The exception to this - the people who would take the $1 offer - were people who had been taught economic theory. It turns out you _could_ screw them over and they'd pat themselves on the backs for being very wise. The "tragedy of the commons" is another one of those parts of standard economic theory that never actually played out in reality - we've got examples from all over the world of communities implementing practices and often entire belief systems that led them to be responsible stewards of shared resources without requiring unilateral ownership of that resource and singular acquisition of the benefits of that stewardship, and yet first on the lips of every modern capitalist when describing why they're at a disadvantage if they're not the ones polluting the water supply is the tragedy of the commons.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2024Hacker News 讨论帖274 分 · 187 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选