AI 与机器学习2025arxiv.org
为什么 Transformer 学不会乘法？
Why can't transformers learn multiplication?
进阶 · 约 60 分钟读完前置知识：Transformer 基础、注意力机制、算法学习/课程学习概念
导读
这篇论文研究 Transformer 为何难以学会多位数乘法这类需要精确符号运算的任务。文章分析了其失败的原因，并探讨了让模型掌握算法式运算的可能路径。
HN 关注点：LLM 算不准乘法是大家熟知的痛点，这篇论文从理论上解释了原因，引发关于语言模型与符号运算能力边界的讨论。
HN 评论精选
这几条评论里：支持 1 质疑 1 补充 2
支持人类语言模型同样不擅长乘法，精确计算依赖符号操作，要求 LLM 擅长乘法是超人类期望
这是我的直觉判断：LLM 是大语言模型，而我大脑里的语言模型也不擅长大规模乘法。我对个位乘法表有基于语言的直觉（略超个位，据我观察已高于同龄人平均水平），但算 283×9284 靠的是符号操作，而我的神经网络在这方面同样糟糕。一台 Commodore PET 远比我的大脑简单，乘法却碾压我；符号系统又在此基础上提升了约 15 个数量级。就算逼我背（训练）多位数乘法到死，我的语言模型也只能略微变好；真正让我进步的是练习符号操作。期待语言模型擅长乘法，是在要求一种超人类的能力，而且没理由相信它会随规模扩展。
jerf查看英文原文
This is a gut impression and I don't deny it, but LLMs are Large Language Models, and in my own brain, my Language Model isn't doing large-scale multiplication. I have a language-based intuition for the sigle-digit multiplication table and a touch beyond (and based on my observations that's already above average for a human Language Model, at least in my age peer group), but it's not my Language Model doing 283 times 9284. That requires a symbolic manipulation model, and in fact I would observe that my personal neural net, for all the things it is amazingly good at, is in fact quite terrible at that sort of multiplication too. A Commodore PET is by all measures vastly, vastly simpler than my brain, but it blows away my multiplication capabilities. And then the symbolic systems tacked on another, what, 15 orders of magnitude from that "blows away my multiplication capabilities"? Depends on how you count, but something like that. You can sit here and force me to recite ("train me on") multi-digit multiplication problems and their result until the day I die, and my language model is only going to get marginally better. It is in practicing my symbolic manipulation that I'm going to get better and faster. It seems to me that expecting a Language Model to be very good at multiplication is asking for a substantially superhuman level of performance from them, and one that we have little reason to believe will scale anyhow. What we need is symbolic manipulation, better than the approximation they achieve when "reasoning". I find it rather ironic to sit here and use the aforementioned 15 orders of magnitude improvement over the Commodore PET to use that level of symbolic manipulation firepower to laboriously recreate a software system that is as bad as we are at multiplication for what may well be the same fundamental reasons... and then have the audacity to complain about it. My metaphorical dude, you did a couple trillion multiplications just to get to this single bad multiplication output... maybe another approach is called for.
补充期待课程式渐进学习架构，并质疑现有训练目标与泛化目标脱节
希望看到更像人类学习方式的架构：从模仿一个字母开始，再到音节、单词、句子，逐步在已有知识上叠加。另外有趣的是，模型的重要目标是泛化，但训练方法优化的却是损失/准确率，训练完才测试泛化来验证。有没有显式最大化泛化的训练方法或课程？
nico查看英文原文
Would love to see an architecture that learned more like humans. Start with just imitating one letter, then a few more, than some syllables, then full words, then sentences, etc. Progressively adding on top of previous knowledge Also, it’s interesting that one of the big goals/measures of models is their capacity to “generalize”, but the training methods optimize for loss/accuracy, and only after training test for generalization to validate Are there training methods/curriculums that explicitly maximize generalization?
补充把模型放进迭代循环、多次尝试，是解决乘法这类迭代任务的方向
之前见过一个实验：让模型反复运行完成代码库跨语言移植，提示词让它给自己留笔记、每次完成一点，然后放进循环里，结果成功了，甚至有一次模型自己编辑文件把自己移出了循环。在我看来，乘法和长除法这类迭代任务很像那个实验。把模型放进循环、给它多次尝试机会，是提升能力的合理方向。
ddingus查看英文原文
A while back I saw a post where people ran a model over and over to accomplish a code base port from one language to another. In their prompt, they told it to leave itself a note and to accomplish something each time. Then they put the model in a loop and it worked. In one instance, a model removed itself from the loop by editing a file or some other basic means. To me, iterative tasks like like multiply and long divide, look an awful lot like the code port experiment. Putting models into loops so they get more than one bite at the task seems to be a logical progression to improve capability.
质疑核心矛盾在于概率性输出与确定性运算的不匹配
因为它们是概率性地产生输出，而乘法是确定性的。为什么这一点大家都想不明白？
carodgers查看英文原文
Because they produce output probabilistically, when multiplication is deterministic. Why is this so hard for everyone?
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2025Hacker News 讨论帖161 分 · 107 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选