AI 与机器学习2024arxiv.org
大语言模型硬件加速：综述与比较
Hardware Acceleration of LLMs: A comprehensive survey and comparison
进阶 · 约 90 分钟读完前置知识：计算机体系结构基础、LLM 推理流程、半导体工艺节点概念
导读
这是一篇综述，系统梳理并比较了用于大语言模型推理/训练的各类硬件加速方案（如 GPU、ASIC、FPGA、存内计算等），并尝试把不同半导体工艺节点下的性能数据用拟合多项式外推到 16nm 这一共同基准，以便横向对比。
HN 关注点：LLM 性能瓶颈正从算力转向内存带宽（“内存墙”），以及为快速迭代的模型定制专用硬件是否划算。
HN 评论精选
这几条评论里：支持 1 补充 2
补充补充“内存墙”历史脉络，指出 LLM 瓶颈正转向内存带宽、存内计算是应对方向，并解释论文的 16nm 归一化方法。
这篇论文背景讲得太少，我补充一些：早在 90 年代就有人注意到 CPU 的 FLOPs 提升速度快于内存带宽。1995 年 William Wulf 和 Sally Mckee 预测这种分化会导致“内存墙”——多数计算会被数据访问而非算术运算卡住。过去 20 年，服务器峰值 FLOPS 每两年增长 3 倍，而 DRAM 带宽和互连带宽每两年只增长 1.6 倍和 1.4 倍。因此 LLM 的训练和推理瓶颈越来越偏向内存带宽，对自回归 Transformer 解码器来说尤其可能成为主导瓶颈。这催生了存内计算（CIM/PIM）这类新技术：直接在内存中的数据上做运算，而不是先把数据搬到 CPU 寄存器，从而改善延迟和功耗，或许能绕开“内存墙”。另外，为了在工艺尺寸不同的情况下比较 ASIC 和 FPGA，论文引用 Stillmaker 和 Baas 的 CMOS 性能缩放公式，把性能外推到 16nm 的共同基准。
refibrillator查看英文原文
This paper is light on background so I’ll offer some additional context: As early as the 90s it was observed that CPU speed (FLOPs) was improving faster than memory bandwidth. In 1995 William Wulf and Sally Mckee predicted this divergence would lead to a “memory wall”, where most computations would be bottlenecked by data access rather than arithmetic operations. Over the past 20 years peak server hardware FLOPS has been scaling at 3x every 2 years, outpacing the growth of DRAM and interconnect bandwidth, which have only scaled at 1.6 and 1.4 times every 2 years, respectively. Thus for training and inference of LLMs, the performance bottleneck is increasingly shifting toward memory bandwidth. Particularly for autoregressive Transformer decoder models, it can be the dominant bottleneck. This is driving the need for new tech like Compute-in-memory (CIM), also known as processing-in-memory (PIM). Hardware in which operations are performed directly on the data in memory, rather than transferring data to CPU registers first. Thereby improving latency and power consumption, and possibly sidestepping the great “memory wall”. Notably to compare ASIC and FPGA hardware across varying semiconductor process sizes, the paper uses a fitted polynomial to extrapolate to a common denominator of 16nm: > Based on the article by Aaron Stillmaker and B.Baas titled ”Scaling equations for the accurate prediction of CMOS device performance from 180 nm to 7nm,” we extrapolated the performance and the energy efficiency on a 16nm technology to make a fair comparison But extrapolation for CIM/PIM is not done because they claim: > As the in-memory accelerators the performance is not based only on the process technology, the extrapolation is performed only on the FPGA and ASIC accelerators where the process technology affects significantly the performance of the systems. Which strikes me as an odd claim at face value, but perhaps others here could offer further insight on that decision. Links below for further reading. https://arxiv.org/abs/2403.14123 https://en.m.wikipedia.org/wiki/In-memory_processing http://vcl.ece.ucdavis.edu/pubs/2017.02.VLSIintegration.TechScale/
补充追问“存内计算”到底指什么硬件形态。
我不太熟悉，这里的“in-memory”是指把 CPU 和 RAM 结合在一起的专用硬件吗？
yjftsjthsd-h查看英文原文
I'm unfamiliar; in this context is "in-memory" specialized hardware that combines CPU+RAM?
支持认为模型迭代太快，专用硬件投入容易打水漂。
存内计算看起来是正确方向，不仅因为性能，还因为为一个很可能几个月内就过时的模型去造 ASIC 或烧 FPGA 根本说不通。
moffkalast查看英文原文
In-memory sounds like the way to go not just in terms of performance, but in that it makes no sense to build an ASIC or program an FPGA for a model that will most likely be obsolete in a few months at best if you're lucky.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2024Hacker News 讨论帖266 分 · 68 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选