AI 与机器学习2018arxiv.org
多项式回归作为神经网络的替代方案
Polynomial Regression as an Alternative to Neural Nets
进阶 · 约 60 分钟读完前置知识：线性回归、神经网络基础、模型评估与基线对比
导读
这篇论文主张用多项式回归（含多项式激活的浅层网络）作为神经网络的替代方案，并在若干小型数据集上做了对比实验，声称效果相当甚至更好。但论文的对比对象只是很小的全连接网络，实验规模也有限，因此其结论能否推广到大规模任务存疑。
HN 关注点：论文标题很宽泛，但实验只用了玩具级数据集和极浅的神经网络，讨论焦点在于这种比较是否有意义、是否夸大了结论。
HN 评论精选
这几条评论里：质疑 3 补充 1
质疑论文只用玩具级实验和微型网络对比，结论被夸大，缺乏与 XGBoost 等强基线及大规模任务的比较。
我不理解这种论文怎么会上 HN 首页。摘要做了非常宽泛的断言，但论文本身只在极小的数据集上用极小的神经网络做玩具实验。没有优秀的 ML 从业者会认为小型浅层全连接网络是所有问题的最佳算法（不信看 Kaggle 结果）。尤其对论文里那种小而简单的数据集，NN 往往不是最佳选择；但对大规模图像分类、密度建模、序列建模等（论文都没测），NN 才是 SOTA。可笑的是论文只把多项式回归和微型神经网络比。我敢打赌，如果加入 XGBoost 等经典 ML 方法的结果，多项式回归会被打得落花流水。还有一些迹象可以判断这类论文不代表 ML 领域：引用缺失（如'众所周知 NN 容易过拟合 [Chollet and Allaire(2018)]……e.g. [?]'）、格式不一致（如第 7 页的表格）、以及在听起来很容易做的实验后面写'还需要更多实证工作来探索这些问题'。
unixpickle查看英文原文
I don't understand how papers like this end up on the homepage of HN. The abstract makes very broad claims, but the paper itself only runs toy experiments on tiny datasets with tiny neural networks. No good ML practitioner believes that tiny, shallow, fully-connected neural networks are the best algorithm for every problem (just look at Kaggle results if you don't believe me). Especially for small, easy datasets like the ones analyzed in this paper, NNs are often not the best choice. However, for large scale image classification, density modeling, sequence modeling, etc. (none of which are tested in the paper), NNs are SOTA. Hilariously, the paper only compares polynomial regression to tiny neural networks. I bet if they had thrown in results from XGBoost or other classical ML techniques, polynomial regression would be blown out of the water. There's also some heuristics you can use to tell when a paper like this isn't necessarily representative of the field of ML: - Missing citations (e.g. "It is well-known that NNs are prone to overfitting [Chollet and Allaire(2018)], which has been the subject of much study, e.g. [?]."). - Inconsistent formatting (e.g. the tables on page 7) - Sentences like "Much more empirical work is needed to explore these issues." following something that sounds like an easy experiment to try.
质疑把两层 12 单元的网络称为'深度学习'是误导，论文误解了深度学习复兴的真正原因。
他们对比的神经网络浅得惊人（两层各 12 个单元，记作 'KF/DN, layers 12,12'），却称之为'深度学习'。2012 年以来深度学习复兴的关键，从来都是几十层隐藏层的深网络（借助 GPU）在多个重要子领域（著名的 ImageNet 2014，以及革新 NLP、AlphaGo）打破纪录、达到 SOTA，而非神经网络能学习这一早已在 90 年代就被认识的事实。这篇论文没有认识到当今人们为何把深度学习当作 ML 工具，做着无意义的比较，因此只能得出无意义的观察。
mxwsn查看英文原文
The neural nets they compare to are remarkably shallow and thin (2 layers of 12 units each, written as "KF/DN, layers 12,12") to be described as "deep learning". The surprising resurgence of deep learning since 2012 is not the observation that neural network models can learn at all -- neural nets with a handful of hidden layers were known to work fairly well since the 90s -- but that deep nets with dozens of hidden layers (enabled by GPUs) broke records and achieved state-of-the-art results on remarkably challenging tasks (famously, ImageNet 2014) in multiple important machine learning subfields (has revolutionized NLP, also AlphaGo). This paper fails to recognize why people care about deep learning as a tool for machine learning in our modern day, and by making meaningless comparisons they restrict themselves to making meaningless observations.
补充补充了一篇相关论文（Neural ODE）供参考。
我长期在想这个问题——我看到并理解大家指出这篇论文可能有点浅——同样地，我桌上放着这篇准备啃：https://arxiv.org/abs/1806.07366，想问问有没有人看过。
MarkMMullin查看英文原文
I've long wondered about this - I see and understand the comments by people pointing out this paper may a bit shallow - in the same sort of vein, I have this on my desk to grind thru - interested if anyone else has looked at it -- https://arxiv.org/abs/1806.07366
质疑多项式近似在原理上成立，但低阶近似差、高阶易爆炸，且需要与最佳 NN 做更严格对比。
核心观察是 NN 的非线性可以用多项式近似。这当然对，但尤其低阶多项式时近似可能很差，或与被逼近函数性质迥异。特别是 ReLU 激活不会像高阶多项式激活那样向无穷爆炸。另一方面，只要网络能训练，没人关心非线性到底是什么，而且有很多工作把层输入保持在零附近（batch normalization），所以多项式爆炸可能不是大问题。我希望看到他们的模型结果与同类最佳 NN 做更认真的比较。我怀疑他们的 NN 字符检测器基本上是失效的。
sdenton4查看英文原文
The core observation here is that NN nonlinearities can be approximated by polynomials. And sure, that's true, but especially with low degree polynomials the approximation might suck, or have very different properties from the actual function in question. In particular, RELU activation doesn't explode towards infinity the way a high degree polynomial activation would. OTOH, no one cares what the nonlinearity actually is so long as the net trains, and there's a lot of effort to keep layer inputs in a neighborhood near zero (batch normalization), so polynomial explosion may not be such an issue. Feel like I would like a more serious comparison of their model's results with best of class NNs. I'm suspicious that their NN character detector was basically failing...
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2018Hacker News 讨论帖137 分 · 46 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选