AI 与机器学习2025arxiv.org
LIMO：推理中的少即是多
LIMO: Less Is More for Reasoning
进阶 · 约 45 分钟读完前置知识：大模型微调、数学推理基准、Qwen 系列模型
导读
LIMO 论文提出“少即是多”的推理假说：只需精心筛选的约 817 条数学训练样本微调，就能大幅激发大模型（基于 Qwen-2.5 Instruct）的数学推理能力。作者认为当代 LLM 在预训练中已蕴含丰富的数学知识，挑战在于“知识激发”而非“知识获取”。
HN 关注点：这一结论与“LLM 难以泛化定理证明”的流行观点相矛盾，以及 817 条样本的惊人效率背后是否隐藏了大量前置智能投入。
HN 评论精选
这几条评论里：支持 1 质疑 1 补充 1
质疑质疑：817 条样本的成效依赖强大的基座模型和用 SOTA 模型筛选数据，并非表面上那么惊人
结果很酷，但值得指出两点：模型是从 Qwen-2.5 Instruct 微调的，其预训练和监督微调中已包含数百万条经过专门筛选的数学样例。另外，为了生成那 817 条“完美”数学样本，他们用 R1 等最先进模型从 1000 万道数学题的初始池中筛选。也就是说，大量智能被用于打造一个信息密度最大化的精炼微调集。我不确定这比直接在 1000 万条初始数据上微调得到同样结果更令人印象深刻还是更不令人印象深刻，但后者显然标题不够吸引人。
highfrequency查看英文原文
Cool result, but worth highlighting two points: - Model is finetuned from Qwen-2.5 Instruct, which includes millions of specially filtered math examples in both pretraining and supervised fine-tuning already. - To generate the perfect 817 math examples for LIMO, they used state of the art models like R1 to filter down from an initial pool of 10 million math problems. In other words, a whole lot of intelligence was used to craft a maximally informative and distilled set of fine-tuning data. It’s not very clear to me if this is more or less impressive than getting the same result by simply fine-tuning on the 10 million initial pool, but I suppose that would make for a worse headline.
补充指出论文结论与“LLM 无法泛化定理证明”的流行论断相互矛盾
我最近读到两个看似矛盾的说法：1) LLM 永远无法泛化定理证明；2) 这篇论文说“当代 LLM 的参数空间中可能已蕴含丰富的数学知识，挑战从知识获取转变为知识激发”。我都不知道该信哪个了！
igleria查看英文原文
I think I've recently read two seemingly contradicting things: 1- LLMs can never generalize theorem proving 2- this paper: "This suggests that contemporary LLMs may already possess rich mathematical knowledge in their parameter space, transforming the challenge from knowledge acquisition to knowledge elicitation" Not sure what is what anymore!
支持用“unreal engine 提示词”类比解释：预训练已具备能力，微调只是激活潜在技能
我的理解是（非专业人士，欢迎指正）：模型在互联网上预训练时确实获得了数学推理所需的大部分技能，但由于其任务是预测整个互联网的下一个词分布，通常不会使用这种能力——就像几年前的图像生成模型，在提示词里加上“unreal engine”能显著提升质量，因为含该标签的图片多是高质量截图，从而把生成分布推向更高质量。所以模型其实已具备大部分能力，只需调整少量连接来激活这种潜在技能，因此少量训练样本就足够了。
hexomancer查看英文原文
Here is how I make sense of it (I have no expertise in this subject, please feel free to correct me if I am wrong): I think when the model is pretrained on the internet, it does gain most of the skills required to do mathematical reasoning, however, since its task is to predict the next word distribution on the entire internet, it does not normally use this ability, since most of the text on the internet is not this type of reasoning text (think of generative image models a few years ago, where appending "unreal engine" to a prompt would significantly improve the quality of the output, the reason was that the model was trained to generate the distribution of the images on the internet, most of them are not particularly impressive, however, since images containing "unreal engine" were usually high-quality screenshots of images, it would also move the distribution of generated images towards higher quality generations). So I think the model already has most of the ability, it just needs to adjust a few connections to actually utilize this latent skill, so it makes sense that a few training examples are enough to adjust the connections to increase mathematical reasoning skills.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2025Hacker News 讨论帖389 分 · 128 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选