AI 与机器学习2024arxiv.org
通过多 token 预测实现更好更快的大语言模型
Better and Faster Large Language Models via Multi-Token Prediction
进阶 · 约 45 分钟读完前置知识：Transformer 基础、语言模型训练目标、自回归解码
导读
这篇论文研究如何让大语言模型在训练时不仅预测下一个 token，而是同时预测未来多个 token。做法是在共享模型主干上为每个位置设置多个预测头，分别预测后续若干 token；作者报告这能提升模型质量和样本效率，并可用于自推测解码来加速推理。
HN 关注点：挑战了标准的 next-token 预测范式，并讨论联合概率、提前规划与推理加速等实际影响。
后来怎么样
Meta 在 Llama 3/4 等后续模型训练中采用了多 token 预测目标，该技术成为大模型预训练的常用改进之一。
HN 评论精选
这几条评论里：质疑 1 补充 2
补充逐 token 边际预测与联合序列概率存在错配，建议直接预测第 n 个未来 token 作为低成本替代。
LLM 难道不考虑直到某个输出长度为止所有 token 组合的概率分布吗？我原以为它们已经这样做了。如果不是，它们效果这么好让我很惊讶。考虑 2-bit 序列预测：00 p=0.36、01 p=0.04、10 p=0.30、11 p=0.30，联合最可能是 00；但只预测下一位得到 0 p=0.40、1 p=0.60，于是会选 1，给再下一位的预测造成次优起点。序列越长，联合分布越难分解成边缘分布，这个问题越突出。想到一个很简单的实验：把交叉熵损失改成只考虑未来第 n 个 token，然后画 LLM 性能随 n 的变化，当前模型相当于 n=1。我猜测，直接预测第 n 个 token 或许能绕开预测未来 1 到 n 个 token 联合分布带来的资源爆炸（随 x^n 增长），因为这会隐式要求更好的数据模型。
Xcelerate查看英文原文
Do LLMs not consider the probability distribution over all combinations of tokens up to a certain output length with regard to sequence prediction? I assumed they did that already. If they don’t, I’m amazed they work as well as they do. Consider 2-bit sequence prediction with the following possible outcomes and associated probabilities: 00: p=0.36 01: p=0.04 10: p=0.30 11: p=0.30 So the most likely 2-bit sequence is 00. But on the basis of predicting the next token (bit) alone, we have: 0: p=0.40 1: p=0.60 which suggests that 1 is the next bit and leads to a suboptimal starting point for predicting the bit after that. The error is even more prominent with longer sequences as the joint probability distribution becomes more unfactorizable into marginal distributions (as I would expect any minimal algorithmic description of real-world data to be). Edit: now that I think about this a bit more, a cool research project that would be really simple to carry out might be to modify the cross-entropy loss function to consider only the nth future token in the text training data, and then plot LLM performance vs n, assuming that for all current LLM models we just have n=1. My hypothesis is that you can mostly bypass all of the resource blow-up involved in predicting the joint probability distribution over the next 1 through n tokens (which scales as x^n) by just predicting the nth token directly, since doing so would implicitly require a better data model (at least for human-generated text; this wouldn’t be the case for all types of data).
质疑提问为何不直接预测句子级语义向量，而仍以单词/token 为单位。
先为这个很天真的问题道歉：假设我们能构造编码整句话含义的向量，那是什么阻止我们训练 LLM 去预测这些向量，而不是预测单个词？
throw310822查看英文原文
Apologies in advance for the super naive question; but assuming that we can create vectors that encode for the meaning of entire sentences, what prevents us from training llms to predict those vectors instead of single words?
补充讨论自回归生成是否逐 token 重新计算，以及能否复用已激活的内部状态/提前规划。
目前 LLM 每输出一个 token 都要从头开始吗？比如问“香蕉为什么是黄色的？”，它回答“Bananas are yellow due to a pigment called bromelain.”。我会以为当它输出 “a” 时，“pigment”和“bromelain”这些概念已经在神经网络里被激活了，因为之后它不能再改口去说“一种让人类把每个弯曲物体都看成黄色的光学错觉”。所以它似乎已经提前规划好要谈 bromelain 这种色素。能否在输出 “a” 时利用已经完成的计算？神经网络状态能不能以某种方式保留下来用于下一个答案？
mg查看英文原文
Currently, LLMs start from scratch for each output token, right? Lets say you ask an LLM What makes bananas yellow? And it replies Bananas are yellow due to a pigment called bromelain. I would think that the concept of "pigment" and "bromelain" are already somehow activated in the neural net when it outputs "a". Because now it can't change its mind anymore and follow up with "an optical illusion that makes humans perceive every bent object as yellow". So it seems to have already planned ahead to talk about the pigment called bromelain. Would it be possible to capitalize on the work that has already been done when the LLM outputs "a"? Could the state of the neural net be somehow preserved for the next answer?
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2024Hacker News 讨论帖302 分 · 128 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选