AI 与机器学习2024arxiv.org
仅需加法：面向高能效语言模型
Addition is all you need for energy-efficient language models
进阶 · 约 35 分钟读完前置知识：浮点数表示、神经网络基础、Transformer 基础
导读
这篇论文提出用浮点数加法（L-Mul 算法）替代元素级浮点乘法来计算神经网络，声称可将元素级张量乘法的能耗降低约 95%，点积能耗降低约 80%，从而提升语言模型的能效。核心思路是用整数加法近似浮点乘法，在精度损失可控的前提下大幅省电。
HN 关注点：这一能耗优化是否真的适用于 LLM 推理，以及为何这类思路长期未被业界深入探索。
HN 评论精选
这几条评论里：质疑 3 补充 1
质疑质疑：Transformer 瓶颈在内存带宽而非乘法，论文的节能数字只针对孤立运算，不代表整体推理能耗。
「可减少 95% 元素级浮点张量乘法能耗、80% 点积能耗」——如果这是针对卷积网络，优化计算会重要得多。但 Transformer 计算量轻、内存压力大，链条中最薄弱的环节是把模型权重搬运到计算核心。文中引用的 95% 和 80% 只是乘法运算本身的能耗降低，并非整个推理流程的降低。
visarga查看英文原文
> can potentially reduce 95% energy cost by elementwise floating point tensor multiplications and 80% energy cost of dot products It this were about convolutional nets then optimizing compute would be a much bigger deal. Transformers are lightweight on compute and heavy on memory. The weakest link in the chain is fetching the model weights into the cores. The 95% and 80% energy reductions cited are for the multiplication operations in isolation, not for the entire inference process.
质疑阴谋论式猜测：Nvidia 资助研究生态，导致去乘法化这类威胁 GPU 需求的方向缺乏探索。
这类说法这几年见过好几次，我有个个人猜测为何没被深入探索：Nvidia 资助了大部分 LLM 相关研究。如果 Transformer 能用加法完全取代浮点乘法，GPU 可能就不再必要，或便宜得多的芯片就够用——那 Nvidia 的万亿美元帝国就会动摇。大学实验室拿 Nvidia 的免费 GPU，自然不愿做让这些 GPU 过时的研究。如果属实，说明我们因企业利益被困在低效的研究路径上。希望我是错的。
ranguna查看英文原文
I've seen this claim a few time across the last couple years and I have a pet theory why this isn't explored a lot: Nvidia funds most research around LLMs, and they also fund other companies that fund other research. If transformers were to use addition and remova all usage of floating point multiplication, there's a good chance the gpu would no longer be needed, or in the least, cheaper ones would be good enough. If that were to happen, no one would need nvidia anymore and their trillion dollar empire would start to crumble. University labs get free gpus from nvidia -> University labs don't want to do research that would make said gpus obsolete because nvidia won't like that. If this were to be true, it would mean that we are stuck on an inificient research path due to corporate greed. Imagine if this really was the next best thing, and we just don't explore it more because the ruling corporation doesn't want to lose their market cap. Hopefully I'm wrong.
补充补充历史：用整数运算模拟浮点乘法是老技术，曾在嵌入式和早期软件中广泛使用。
很多年前浮点运算对 Intel CPU 很贵时，程序员用整数技巧绕过。Forth 之父 Chuck Moore 演示过把 1.6×4.1 变成 16×41 的整数运算，最后再把小数点放回正确位置；只要数值范围不超限就可行，对每秒多次计算 10 位精度模拟读数的嵌入式芯片很有效。我还记得一位微软工程师说过，Streets and Trips 也曾把浮点数和计算塞进只保留所需精度的打包整数格式，在当时更快也更容易压缩到 CDROM 上。
shrubble查看英文原文
I remember that many years ago, when floating point computation was expensive for Intel CPUs to do, there were multiple ways that programmers used integer trickery to work around this. Chuck Moore of Forth fame demonstrated taking the value, say 1.6 multiplied by 4.1 and doing all the intermediate calculations via integers (16 * 41) and then formatting the output by putting the decimal point back in the "right place"; this worked as long as the range of floating point values was within a range that multiplying by 10 didn't exceed 65536 (16 bit integers), for instance. For embedded chips where for instance, you have an analog reading with 10 bits precision to quickly compute multiple times per second, this worked well. I also recall talking many years ago with a Microsoft engineer who had worked with the Microsoft Streets and Trips program (https://archive.org/details/3135521376_qq_CD1 for a screenshot) and that they too had managed to fit what would normally be floating point numbers and the needed calculations into some kind of packed integer format with only the precision that was actually needed, that was faster on the CPUs of the day as well as more easily compressed to fit on the CDROM.
质疑质疑研究优先级：认为在 LLM 能力尚未成熟阶段过早优化能效是本末倒置。
我不想当那个提这个问题的人，但难道不应该先让 LLM 工作好，再考虑优化吗？用汽车类比：这就像在早期汽车时代操心让车少烧煤。这是徒劳的。
A4ET8a8uTh0查看英文原文
Uhh.. I hate to be the one to ask this question, but shouldn't we be focused on making LLMs work well first and then focused on desired optimizations? Using everyone's car analogy, it is like making sure early cars are using lower amount of coal. It is a fool's errand.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2024Hacker News 讨论帖334 分 · 126 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选