AI 与机器学习2018arxiv.org
深度学习所需的矩阵微积分（2018）
The matrix calculus you need for deep learning (2018)
入门 · 约 60 分钟读完前置知识：单变量微积分、线性代数基础
导读
这篇论文（由 Jeremy Howard 等人撰写）系统梳理了深度学习所需的矩阵微积分知识，重点澄清了分子布局（denominator layout）与导数布局约定等容易混淆的记号问题，并给出反向传播等场景下的求导规则。它更像一份教学性参考资料，帮助读者从单变量微积分过渡到矩阵求导。
HN 关注点：HN 读者长期关注深度学习的数学基础，这篇多次被 repost 的教程常引发关于求导记号约定和教学方式的讨论。
HN 评论精选
这几条评论里：质疑 3 补充 1
补充掌握几条基本规则加维度匹配，就能应付大部分矩阵求导
如果你懂单变量微积分和基础线性代数，通常只需记住：f: R^n -> R 的导数是向量；f: R -> R^n 的导数是向量；f: R^m -> R^n 的导数是 n×m 的雅可比矩阵；f(x)=x^tAx 的导数是 (A+A^t)x；标量对向量的梯度（二阶导）是对称的海森矩阵。然后套用单变量的求导法则（乘积法则、链式法则等），再调整结果使维度匹配即可。遇到行列式这类复杂函数，需要时查一下导数就行。
yellowcake0查看英文原文
If you know single variable calculus and basic linear algebra, then you can often muddle through by just remembering that the derivative of, (1) f: R^n -> R is a vector (2) f: R -> R^n is a vector (3) f: R^m -> R^n is an n x m matrix (the Jacobian) (*) f(x)=x^tAx is f'(x)=(A + A^t)x (this is an example of (1)) (**) and that the derivative (gradient) of (1) gives you (3) with m=n, and in this case the derivative of (3) will be symmetric (the Hessian). then just do your best to apply the single variable rules of differentiation (product rule, chain rule, etc.), and then mess around a bit with the result until all the dimensions match up and such that your result matches the appropriate case (1)-(3). For any more complicated functions you encounter such as the determinant, you can just look up its derivative when needed.
质疑指标记号比矩阵布局约定更简洁强大，能处理更一般的求导
大家真的这么做吗？用指标记号加爱因斯坦求和约定不是更简单强大吗？只需记两条规则：dx_i/dx_k = [i==k]，以及 (AB)_ij = A_ik B_kj。以 f(x)=x^T A x 为例，求导立即得到 A_kj x_j + x_i A_ik。对 A 求导也一步得出 df/dA_kl = x_k x_l，而文章的方法甚至无法直接处理这种情况。
jules查看英文原文
Do people do it this way? Isn't using index notation + Einstein summation convention way easier and more powerful? You only need to remember two rules: 1. dx_i/dx_k = [i==j] where [i==j] is 0 if i != j and 1 if i == j 2. (AB)_ij = A_ik B_kj You don't even need the second rule if the function you want to differentiate is in index notation in the first place. The example of the other comment: f(x) = x^T A x = x_i A_ij x_j df/dx_k = d/dx_k (x_i A_ij x_j) = [i==k] A_ij x_j + x_i A_ij [j==k] = A_kj x_j + x_i A_ik. You can also skip the intermediate [i==k] step and immediately set the indices of other factors in a term equal. For example, differentiating with respect to A is immediate, even though the method from the article can't even handle it directly: df/A_kl = x_k x_l
质疑Karpathy 认为矩阵微积分并非必需，反向传播用标量链式法则即可实现
深度学习根本不需要矩阵微积分。micrograd（https://github.com/karpathy/micrograd）用 100 行代码实现了神经网络的反向传播，就是证明。其余的都只是向量化而已。
karpathy查看英文原文
You do not need any matrix calculus for deep learning and micrograd (https://github.com/karpathy/micrograd), which implements backpropagation for neural nets in 100 lines of code, is a proof. Everything else is just vectorization.
质疑质疑这类教学性文章是否适合发在 arXiv 上
我现在不会再把 arXiv 只和研究论文联系在一起了，看来上面塞了不少这类所谓的“白皮书”。不过我感谢作者的努力，它更像一本教科书的构想。arXiv 似乎也欢迎这类内容，很高兴看到研究平台上出现更多教学文档，但我觉得教学和研究分开是有原因的。
anonytrary查看英文原文
I will stop associating arxiv with solely research papers now. I guess they getting filled with lots of these so-called "whitepapers". I appreciate the effort of this paper, though, it seems more like an idea for a textbook. It even seems like arxiv is welcoming this what with their "Computers > Learning" breadcrumb, good to see more types of educational documents in the research realm, but I feel that teaching and research are separate for a reason.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2018Hacker News 讨论帖26 分 · 4 评
- 2021Hacker News 讨论帖168 分 · 40 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选