AI 与机器学习2024arxiv.org
没有指数级数据就没有“零样本”
No "Zero-Shot" Without Exponential Data
进阶 · 约 45 分钟读完前置知识：机器学习基础、零样本泛化、CLIP 与扩散模型
导读
这篇论文检验图像-文本和文本-图像模型（如 CLIP、Stable Diffusion）是否真能零样本泛化。作者整理 4000+ 概念，并在约 300 万到 4 亿预训练样本的规模上测试 34 个模型，发现概念越稀有、在预训练数据中出现频率越低，模型表现越差；性能随概念频率指数增长而线性提升，因此所谓“零样本”泛化并不成立。作者还发布长尾基准 “Let it Wag!” 供后续评测。
HN 关注点：挑战了“零样本泛化”叙事，并由此讨论 AI 炒作、数据耗尽以及人类少样本学习与机器学习差距。
HN 评论精选
这几条评论里：支持 1 质疑 1 补充 1
支持详细解读论文：概念频率决定模型表现，所谓零样本泛化不成立，并发布长尾基准。
这值得上首页。作者问图像到文本和文本到图像模型（如 CLIP 和 Stable Diffusion）是否真正具备零样本泛化能力。为回答这个问题，他们整理了一份 4000+ 概念的列表，并测试 34 个不同模型在不同预训练规模（约 300 万到 4 亿样本）下分类或生成这些概念的效果。他们发现，每个概念上的模型性能会随着该概念在预训练数据中的频率指数增长而线性提升，也就是说概念越稀有，模型越不可能真正/正确学到它——这意味着不存在“零样本”泛化。作者还发布了一个长尾测试数据集，并巧妙命名为 “Let it Wag!” benchmark，让其他研究者能自行观察当前模型在日益稀有的长尾概念上的表现。去读全文，至少读引言。它清晰、简洁、写得好。
cs702查看英文原文
This deserves to be on the front page. The authors ask whether image-to-text and text-to-image models (like CLIP and Stable Diffusion) are truly capable of zero-shot generalization. To answer the question, the authors compile a list of 4000+ concepts (see paper for details on how they compile the list of concepts), and test how well 34 different models classify or generate those concepts at different scales of pretraining, from ~3M to ~400M samples. They find that model performance on each concept scales linearly as the concept's frequency in pretraining data grows exponentially, i.e., the rarer the concept the less likely it is actually/properly learned -- which implies there is no "zero-shot" generalization. The authors also release a long-tail test dataset that they cleverly name the "Let it Wag!" benchmark to allow other researchers to see for themselves how current models perform on the long tail of increasingly rare concepts. Go read the whole thing, or at least the introduction. It's clear, concise, and well-written.
补充质疑人类少样本学习与机器学习依赖海量数据的巨大差距。
不想跑题，但……我给家人解释 AI 时用分类举例，具体用猫和狗。我用教幼儿“这是猫、那是狗”的类比，本质是重复。起初他们会混淆，父母会在孩子把狗当猫时说“不，那是狗”。但孩子学猫狗区别只需要各看几张图，之后基本就会了。那么为什么机器学习要数百万甚至数十亿张图才能说“这是猫”，而人类几张（最多可能 100 张）就行？为什么机器学习还做不到？我是多年开发者但不在 AI 领域，所以问个 ELI5 问题 :) 补充：这是否与孩子看到静态图还是活猫有关？活猫可能被大脑当作成千上万帧静态图处理？
ItsBob查看英文原文
Not to derail this conversation but... When I'm explaining AI stuff to family, the example I use is classification and I specifically use cats and dogs. I use the analogy of how you teach a toddler that this is a cat and that is a dog. Essentially repetition. And at first they get them mixed up and the parent will say "no, that's a dog" when they think it's a cat and so on. But essentially, for a child learning the difference between a cat and a dog you only need to show them a handful of each and they'll generally get it from that point on. That being said, why does it take ML millions (or billions) of images to be able to say "that's a cat" when a human does it on a handful (might be up to, say 100 but my point stands). Why can ML not do that yet? I'm a dev for many years but not in AI, hence my ELI5 question :) Edit: If the answer is massively long and complicated, perhaps if you could point me to some text (book, paper etc.) and I can read at my leisure. Edit2: I just thought of something. Is it related to whether the child sees a still image or a live cat? So, for example, a still image is a single example of a cat standing in a particular position etc, whereas a moving, live cat, would be interpreted by the brain as many many still images, all processed individually? The end result being that, in fact the child, when seeing a live cat, actually sees thousands or millions of still images of the cat? It just popped into my head there :D
质疑认为耗尽互联网数据、投入巨大后仍只得到平庸 AI，质疑投入产出。
这感觉是当前 AI 炒作最糟的结果。我们基本已经把整个互联网扒下来喂给模型，过程中花了数十亿美元。这几乎是目前能获得的最大数据集，而且由于互联网被 AI 污染日益加剧且现在加速，很可能也是未来能存在的最大数据集。尽管如此，我们得到的只是不算完全无用但仍相当糟糕的 AI？要是从没做过这事，我们可能更好。
crote查看英文原文
This feels like the worst possible outcome of the current AI hype. We've essentially been ripping off the entire internet and feeding it to the models already, spending many billions of dollars in the process. It's pretty much the largest possible dataset you can currently get, and due to the ever-increasing and now rapidly accelerated AI poisoning of the internet most likely the largest possible dataset which will ever exist. All that and all we're getting out of it is not-entirely-useless but still quite crappy AI? We would've been better off if we had never done this.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
- 视频与演讲
- 评论区提到的论文
历次讨论
- 2024Hacker News 讨论帖187 分 · 118 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选