数学2015projecteuclid.org编辑精选
统计建模：两种文化（2001）
Statistical Modeling: The Two Cultures (2001)
进阶 · 约 60 分钟读完前置知识：统计学基础、回归分析、模型评估
导读
这篇论文讨论统计学中两种建模文化：一种假设数据由某种随机模型生成，侧重推断与系数解释；另一种把数据生成机制视为未知，侧重预测精度，主张使用算法建模。Breiman 认为统计学过度偏向前者，并以 CART 等方法为例倡导更重视样本外预测。
HN 关注点：统计学与机器学习的范式差异，以及样本外预测、R^2 等传统指标是否可靠；评论还补充了 Donoho 的后续综述和标题典故。
后来怎么样
这篇论文成为统计学与机器学习文化分野的经典论述，推动了样本外预测和算法建模在统计实践中的能见度，并引发大量后续讨论与综述。
HN 评论精选
这几条评论里：支持 1 补充 2 亲历 1
亲历高度评价论文，并补充 Donoho 的后续综述，提醒两种文化的二分法已随时间软化。
这是一篇很棒的小论文——还附有评论和作者回应！它刚发表时我曾在读书小组里做过分享，因为我太喜欢了。它总值得重读，因为 Breiman 是有用概率建模与洞察力方面的英雄。应当记住它是其时代的反映，它提出的二分法多年来已被软化。另一篇更近期的论文，在更广背景下重新审视其中一些相同趋势，是 David Donoho 的：https://courses.csail.mit.edu/18.337/2015/docs/50YearsDataScience.pdf 。强烈推荐。HN 上也有不错的评论：https://news.ycombinator.com/item?id=10431617
mturmon查看英文原文
This is a great little paper -- with comments and rejoinder! I presented it to a reading group back when it appeared, I enjoyed it so much. Always worth re-reading because Breiman is such a hero of useful probabilistic modeling and insight. One should remember that it is a reflection of its time, and the dichotomy it proposed has been softened over the years. Another paper, more recent, and re-examining some of these same trends in broader context, is by David Donoho: https://courses.csail.mit.edu/18.337/2015/docs/50YearsDataScience.pdf Highly recommended. Pretty good HN comments at: https://news.ycombinator.com/item?id=10431617
支持强调样本外预测精度的重要性，并批评 R^2 等样本内指标和 NHST 传统。
这是一篇很棒的论文。非常长，但每一部分都值得。顺便说，这里有一篇近期关于该论文的博客文章：http://duboue.net/blog27.html 。我带走的关键洞见之一是：在统计学的回归任务中，使用样本外预测精度作为指标很重要——就像机器学习中那样。STATS 101 的标准最佳实践是计算 R^2 系数（基于样本内数据），这类似于报告训练数据上的误差估计（样本内预测精度）。依我看，统计学是最迷人、最有用的研究领域之一，应用无数。要是我们能轻易区分哪些是“遗留代码”、哪些是根本性的就好了……另见近期文章 https://www.gwern.net/Everything ，它指出零假设显著性检验（NHST）的局限，这也是 STATS 101 的另一支柱。
ivan_ah查看英文原文
This is a great paper. Very long, but worth every bit of it. BTW, here is a recent blog post about the paper: http://duboue.net/blog27.html One of the key insights I took away was the importance of using out-of-sample predictive accuracy as a metric for regression tasks in statistics—just like in ML. The standard best practices in STATS 101 is to compute R^2 coefficient (based on data of the sample), which is akin to reporting error estimates on your training data (in-sample predictive accuracy). IMHO, statistics is one of the most fascinating and useful fields of study with countless applications. If only we could easily tell apart what is "legacy code" vs. what is fundamental... See this recent article https://www.gwern.net/Everything the points out the limitations of Null Statistical Hypothesis Testing (NHST), another one of the pillars of STATS 101.
补充指出标题典故来自 C.P. Snow 的“两种文化”。
标题参考的是 C.P. Snow 那篇关于人文与科学之间分裂的著名文章：https://en.wikipedia.org/wiki/The_Two_Cultures
astazangasta查看英文原文
The title is a reference to this famous essay by C.P. Snow about a split between the humanities and science: https://en.wikipedia.org/wiki/The_Two_Cultures
补充解释 Breiman 的核心论点：传统回归系数解释有局限，复杂数据更需准确预测，CART 等算法建模可替代。
简言之：在一些重要的统计建模应用中，传统回归分析效果不佳或不适用。使用回归的原因之一是把回归系数当作对应变量重要性的指南。嗯，即便从偏导数角度看，这也一直有点可疑。在一些重要应用里，例如某些非常复杂的生物医学数据，真正需要的不是过度强调系数，而是一个能给出准确预测的模型。为了得到这样的预测，我们可以不再只强调传统回归分析及其系数解释，而使用更一般的模型构建手段。相关的旧方法包括用于带限平滑与插值的傅里叶方法、多元最小二乘样条插值，以及分类数据分析，例如逻辑回归与可追溯到社会科学数学的对数线性模型。然后 Breiman 等人做了分类与回归树——即 CART，有软件。简言之，建树：从根节点开始，把输入数据分成两个集合，每个集合放入根节点的一个子节点，分别拟合每个集合。然后类似地分裂这两个节点。这样也许能得到好的数据拟合和好的预测值。然而，失去了考虑……
graycat查看英文原文
In short: In some important uses of statistical model building, traditional regression analysis doesn't work or fit very well. One of the reasons to use regression was to look at the regression coefficients as guides to the importance of the corresponding variables. Well, that was always a bit fishy anyway for more than just a differential, i.e., partial derivative, view. In some important applications, e.g., with some really complicated biomedical data, instead of so much emphasis on the coefficients, what is really needed is a model with some accurate predictions. So to get such predictions, we can drop the emphasis on just traditional regression analysis and interpreting its coefficients and, instead, use much more general means of model building. Relevant old approaches have been Fourier methods for band limited smoothing and interpolation, multivariate least squares spline interpolation, and categorical data analysis, e.g., logistic regression with log-linear going way back in the math for the social sciences. Then Breiman and others did Classification and Regression Trees -- a.k.a., CART for which there is software. In short, build a tree: Start at the root node. Take the input data and partition it into two collections, put each collection in a node that is a child of the root node, and fit each collection separately. Then split those two nodes similarly. Then maybe have a good fit to the data and good predicted values. However, have lost the considerable advantages of traditional regression analysis, but also didn't have or use the sometimes severe assumptions of that old analysis. With traditional regression, had a lot in statistical hypothesis tests, etc., to evaluate the quality of the work, but with the tree, etc. don't have such things. So, for the tree, evaluate the quality by seeing how accurate the model is on a second collection of data, test data, statistically similar to the first collection used to find the tree. If the model is accurate on the test data, then maybe have some good work. Else try again and then do some more statistical work to avoid a case of just getting a spurious fit from just trying until got lucky. Since I have a lot of respect for Breiman, I'm willing to take seriously whatever he wrote.
历次讨论
- 2015Hacker News 讨论帖35 分 · 4 评
- 2017Hacker News 讨论帖19 分 · 1 评
- 2019Hacker News 讨论帖200 分 · 31 评
同领域还值得读
一种新的相关系数
A New Coefficient of Correlation
提出了一种新的相关性系数（由 Sourav Chatterjee 提出），能够检测变量之间任意形式（不限于线性）的相关关系，且计算简单高效，主要基于排序和距离计算。
数学2021 · 682 分 · 158 评 · 引用 1进阶 · 约 60 分钟编辑精选数学的两种文化
The Two Cultures of Mathematics (2000)
这是数学家 Timothy Gowers 于 2000 年发表的一篇随笔，他把数学研究大致分成两种文化：一类是“解题者”，热衷于攻克具体难题、发展技巧（如组合数学）；另一类是“理…
数学2009 · 307 分 · 102 评入门 · 约 35 分钟编辑精选重新定义统计显著性
Redefine statistical significance
提议将统计显著性阈值从常用的 p<0.05 收紧到 p<0.005，以降低假阳性率、缓解可重复性危机。
数学2017 · 303 分 · 114 评入门 · 约 45 分钟编辑精选通信的数学理论 (1948)
A Mathematical Theory of Communication (1948)
这是香农 1948 年的经典论文《通信的数学理论》，奠定了信息论的基础。
数学2016 · 169 分 · 53 评入门 · 约 90 分钟编辑精选Hamiltonian Monte Carlo 的概念性介绍
A Conceptual Introduction to Hamiltonian Monte Carlo
是对哈密顿蒙特卡洛（HMC）的概念性导论：它借助哈密顿动力学和统计物理的直觉，把目标分布视为势能景观，引入辅助动量并利用梯度信息沿能量等高线高效采样。
数学2017 · 155 分 · 33 评 · 引用 8进阶 · 约 90 分钟编辑精选