AI 与机器学习2021arxiv.org
正则化就够了：简单神经网络即可在表格数据上表现出色
Regularization is all you need: simple neural nets can excel on tabular data
进阶 · 约 60 分钟读完前置知识：梯度提升树基础、神经网络训练基础、正则化概念
导读
这篇论文研究深度学习在表格数据上为何长期不敌梯度提升树（GBDT）的问题，提出用一组简单正则化技巧（如 dropout、标签平滑等“鸡尾酒”）调优的普通 MLP 即可在表格基准上达到甚至超越 GBDT 的表现。结论表明表格数据未必需要专门的神经架构，正则化才是关键。
HN 关注点：表格数据是工业界最常见的场景，而 GBDT 一直是默认赢家，这篇论文挑战了这一共识，讨论焦点是正则化后的简单神经网络能否真正取代树模型。
HN 评论精选
这几条评论里：质疑 3 补充 1
质疑实践中 GBDT 仍是首选，因为神经网络的超参调优成本太高
有意思，但论文自己也承认，在大多数实际应用中人们可能仍会选梯度提升树——因为即使是用上正则化鸡尾酒的 MLP，也需要投入大量算力做超参数调优。想基于表格数据快速搭起一个可用模型，GBDT 依然无可匹敌。
amilios查看英文原文
This is interesting, but the paper still notes that in most "real-life" applications people will likely still prefer gradient-boosted trees, just because you need to allocate significant computation to hyperparameter tuning even in the case of the MLP-with-regularization-cocktail. For just getting something off the ground quickly based on tabular data, GBDTs are still unbeatable.
补充没有空间/时间结构的数据本来就该用传统方法，结论并不意外
图像有空间相关性，所以用卷积网络；时间序列有时间相关性，所以用循环网络；没有明显相关结构的表格数据，用传统机器学习方法（ANN、SVM、决策树、LR、KNN）就好。从 2006 年左右关注这个领域的人对此并不意外——深度卷积网络一直被视为针对具有多尺度局部相关性数据的一种特例。
periheli0n查看英文原文
Images -> correlation in space -> Deep convolution networks Time series -> correlation in time -> Recursive networks Tabular data without clear correlation structure -> good old ML (ANN, SVM, DT, LR, KNN). This is obvious when following the field since 2006 or so. Deep Convolutional Networks were considered a special case for data with local correlations at a hierarchy of spatial scales. Same for RNNs in time, although they came much later (when was the LSTM rediscovery again? 2016?) For most data without clear spatial or temporal structure to exploit, the good old ML techniques will work just fine.
质疑质疑论文的宏大叙事：基准上的成功不等于真实任务上的成功
论文称表格数据集是深度学习“最后一座未被攻克的城堡”，这个说法有点夸大。更大的问题在于混淆了地图与疆域：ML 领域通过构造各种基准并在其上取得好成绩来体现进步，但在基准上成功不等于在真实任务上成功——NLP 测试不等于掌握真实语言，驾驶基准不等于掌握真实驾驶。基准成功并非毫无意义（围棋就是实验室成功转化为实战的例子），但对于驾驶或语言这类需要持续“合理性”的活动，情况要复杂得多。
joe_the_user查看英文原文
From paper: Tabular datasets are the last "unconquered castle" for deep learning, with traditional ML methods like Gradient-Boosted Decision Trees still performing strongly even against recent specialized neural architectures. My but this statement seems more than a little grandiose. Never mind that XGBoost still does well on a substantial portion of ML challenges (supposedly). The bigger problematic is that there's a confusion of maps and territories in this way of talking of machine learning. The field of ML has made a certain level of palpable progress by having created a number of challenges and benchmarks and then doing well on them. But success on a benchmark isn't necessarily the same as a success the "task" broadly. An NLP test doesn't imply mastering real language, a driving benchmark doesn't imply master over the road driving. etc. Notably, success on a benchmark also "isn't nothing". In a situation like the game of go, the possibilities can be fully captured "in the lab" and success at tests indeed became success against humans. But with driving or language, things are much more complicated. What I would say is that benchmark success seems to produce at least a situation where the machine can achieve human-like performance for some neighborhood (or tile or etc) limited in time, space and subject. Of course, driving is the poster-child for the limitations of "works most of the time" but lot of "intelligent" activities require an ongoing thread of "reasonableness" aside from having an immediate logic. Anyway, it would be nice if our ML folks looked at this stuff more as a beginning than as a situation where they're poised on success.
质疑缺少关键对照实验：传统方法加上同样的正则化后是否仍被击败？
论文缺少一个对照组：把这套“正则化鸡尾酒”同样应用到 XGBoost 等传统方法上效果如何？充其量只能证明加了正则化的神经网络能打败没加正则化的传统方法，要公平比较，两种方法必须能使用同样的正则化手段。
nightcracker查看英文原文
Paper is missing the control: how good is this 'cocktail of regularization' when applied to traditional methods like XGBoost? At best you can claim the result here that neural networks with regularization methods can beat traditional methods without it, but to be apples to apples both methods must have access to the same 'cocktail of regularization'.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2021Hacker News 讨论帖216 分 · 93 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选