数学2022tandfonline.com
过去 50 年最重要的统计学思想有哪些？
What are the most important statistical ideas of the past 50 years?
进阶 · 约 40 分钟读完前置知识：统计学基础、贝叶斯推断、模型评估
导读
这篇论文回顾过去 50 年统计学中最重要的思想，讨论哪些方法或概念对统计实践产生了最大影响，并给出作者的评选与排序。文章引发读者从验证集、MCMC、统计教学和经典滤波方法等角度补充自己的候选清单。
HN 关注点：“哪些统计思想真正重要”的评选，并延伸到机器学习中的 holdout 验证、贝叶斯与频率学派之争，以及统计教学是否该更重计算和模拟。
HN 评论精选
这几条评论里：补充 4
补充留出集验证是过去 50 年极重要的统计思想，能绕开模型选择的认识论困境，并应与可证伪性联系起来。
留出集验证。我 1990 年代学统计时，学的是假设检验（还有 p 值钓鱼等麻烦）和贝叶斯推断（很美妙，直到你要为模型空间发明先验，比如神经网络架构的先验）。这两套体系在回答“我该用哪个模型”时都容易陷入认识论纠缠。留出集验证如此干净简单，只要有大数据就很容易用，还绕开了频率派和贝叶斯的纠缠，所以如今在 ML 中很普遍。它也让统计推断与 Popper 的科学可证伪性一致：科学家用新实验数据检验模型，数据科学家用性质不同的留出集检验模型。注意别用随机打乱得到留出集，因为那不是 Popper 所说的“真正有风险的验证”。文章提到 Breiman 以预测而非建模为基础的统计观；Breiman 很重视留出集评估，但他的“预测”概念不够通用，容纳不了生成式建模（如 GPT、GAN）。更好的说法是“在留出集上评估模型拟合”，这样预测式和生成式建模都能涵盖。
datastoat查看英文原文
Validation on holdout sets. When I was a student in the 1990s, I was taught about hypothesis testing (and all the hassle of p-fishing etc.), and about Bayesian inference (which is lovely, until you have to invent priors over the model space -- e.g. a prior over neural network architectures). These are both systems that tie themselves in epistemological knots when trying to answer the simple question "What model shall I use?" Holdout set validation is such a clean simple idea, and so easy to use (as long as you have big data), and it does away with all the frequentist and Bayesian tangle, which is why it's so widespread in ML nowadays. It also aligns statistical inference with Popper's idea of scientific falsifiability -- scientists test their models against a new experimental data, data scientists can test their model against qualitatively different holdout sets. (Just make sure you don't get your holdout set by shuffling, since that's not what Popper would call a "genuine risky validation".) The article mentions Breiman's "alternative view of the foundations of statistics based on prediction rather than modeling". Breiman does make a big deal of evaluation on holdout sets; but his "prediction" idea isn't general enough, since it doesn't accommodate generative modelling (e.g. GPT, GANs). I think it's better to frame ML in terms of "evaluating model fit on a holdout set", since that accommodates both predictive and generative modelling.
补充评论者给出自己的重要统计思想榜单，认为 MCMC、HMM/Viterbi、VC 理论与 SVM 应并列第一。
凭记忆快速列一下，不深挖统计史书籍：并列第一：MCMC 与 Metropolis–Hastings 算法；隐马尔可夫模型与在线性时间内求最可能序列的 Viterbi 算法；Vapnik–Chervonenkis 统计学习理论与 SVM。第四：Edwin Jaynes 用最大熵构造先验（边界情况：1957 年）。荣誉提名：Breiman 等人的 CART 算法（以及 Quinlan 的 C5.0 扩展）；Box–Jenkins 方法（用 ARMA/ARIMA 找时间序列模型对历史值的最佳拟合）。相比之下，20 世纪初要丰产得多：Kolmogorov、Fisher、Gosset、Aitken、Cox、de Finetti、Kullback、Pearson 家族、Spearman 等。
jll29查看英文原文
Off-the-cuff, i.e. without digging deeply into a set of history of statistics books: Tied 1st place: • Markov chain Monte Carlo (MCMC) and the Metropolis–Hastings algorithm • Hidden Markov Models and the Viterbi algorithm for most probable sequence in linear time • Vapnik–Chervonenkis theory of statistical learning (Vladimir Naumovich Vapnik & Alexey Chervonenkis) and SVMs 4th place: • Edwin Jaynes: maximum entropy for constructing priors (borderline: 1957) Honorable mentions: • Breiman et al.'s CART (Classification and Regression Trees) algorithm (and Quinlan's C5.0 extension) • Box–Jenkins method (autoregressive moving average (ARMA) / autoregressive integrated moving average (ARIMA) to find the best fit of a time-series model to past values of a time series) (The beginning of the 20th century was much more fertile in comparison - Kolmogorov, Fisher, Gosset, Aitken, Cox, de Finetti, Kullback, the Pearsons, Spearman etc.)
补充统计教学应利用廉价计算，从数据模拟和 bootstrap 入手，而不是只教公式和 p 值。
这是个枝节问题，但我想知道统计教学能怎么改，尤其是大学甚至高中教的“给科学家的统计”。我马上想到的是，我学统计时是用公式把数据集压缩成答案，比如零假设显著性检验和令人头疼的 p 值。当时必须这样，因为计算机还不普及。如今计算便宜又容易。我在想能否换种方式学统计，比如先从玩数据、模拟随机数、画图开始；能否主要用 bootstrap 教零假设检验，把公式作为补充？我知道统计是形式化的数学分支，有定理和证明，我也上过那门课。但上“给科学家的统计”的学生从没见过那一面。理解公式却没看过证明，就像想学大一物理却不学微积分。
analog31查看英文原文
This is a side issue, but I wonder what changes could be made to the teaching of statistics, most importantly "stats for scientists" taught in college and even high school. What occurs to me right off the bat is that I learned stats by the use of formulas to boil data sets down to answers. For example, null hypothesis significance testing and the dreaded p-value. We had to do it this way, as widespread availability of computers was still on the horizon. Today, computation is easy and cheap. I wonder if we could learn stats in a different way, perhaps starting by just playing with data, and simulated random numbers, graphing, and so forth. Can something like null hypothesis testing be taught primarily through bootstrapping, with the formulas introduced as an aside? Yes I know that statistics is a formal branch of math, with theorems and proofs. I took that class. But the students who take "stats for scientists" don't ever see that side of it. Understanding the formulas without seeing the proofs is like trying to learn freshman physics without calculus.
补充Kalman 滤波虽略超 50 年，但多传感器融合思想影响巨大，值得列入重要统计思想。
Kalman 在 1960 年发表了他的滤波器，略超过 50 年，但鉴于影响巨大，我觉得值得提。用多个有噪声的传感器得到比任何单个传感器更准确的读数，这一思想支撑了各种自主系统。基本上所有使用传感器的设备（如今几乎每个设备都是）都因此变得更好。
ModernMech查看英文原文
Kalman published his filter in 1960... a little over 50 years but I'd say it's worthy to mention given its huge impact. The idea that we can use multiple noisy sensors to get a more accurate reading than any one of them could provide enables all kinds of autonomous systems. Basically everything that uses sensors (which is essentially every device these days) is better due to this fact.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2022Hacker News 讨论帖396 分 · 98 评
同领域还值得读
一种新的相关系数
A New Coefficient of Correlation
提出了一种新的相关性系数（由 Sourav Chatterjee 提出），能够检测变量之间任意形式（不限于线性）的相关关系，且计算简单高效，主要基于排序和距离计算。
数学2021 · 682 分 · 158 评 · 引用 1进阶 · 约 60 分钟编辑精选数学的两种文化
The Two Cultures of Mathematics (2000)
这是数学家 Timothy Gowers 于 2000 年发表的一篇随笔，他把数学研究大致分成两种文化：一类是“解题者”，热衷于攻克具体难题、发展技巧（如组合数学）；另一类是“理…
数学2009 · 307 分 · 102 评入门 · 约 35 分钟编辑精选重新定义统计显著性
Redefine statistical significance
提议将统计显著性阈值从常用的 p<0.05 收紧到 p<0.005，以降低假阳性率、缓解可重复性危机。
数学2017 · 303 分 · 114 评入门 · 约 45 分钟编辑精选统计建模：两种文化（2001）
Statistical Modeling: The Two Cultures (2001)
讨论统计学中两种建模文化：一种假设数据由某种随机模型生成，侧重推断与系数解释；另一种把数据生成机制视为未知，侧重预测精度，主张使用算法建模。
数学2015 · 254 分 · 36 评进阶 · 约 60 分钟编辑精选通信的数学理论 (1948)
A Mathematical Theory of Communication (1948)
这是香农 1948 年的经典论文《通信的数学理论》，奠定了信息论的基础。
数学2016 · 169 分 · 53 评入门 · 约 90 分钟编辑精选