AI 与机器学习2019arxiv.org
使用基于超立方体的拓扑覆盖进行神经网络一次性训练
One-Shot Training of Neural Networks Using Hypercube-Based Topological Coverings
进阶 · 约 60 分钟读完前置知识：机器学习基础、神经网络基础、拓扑学基础
导读
这篇论文提出一种基于超立方体拓扑覆盖的神经网络构造方法，试图通过几何构造一次性（one-shot）确定网络参数，而非传统的迭代训练。论文在 MNIST 等小数据集上报告了准确率（如 200/2000 样本下约 62%/76%），声称该方法有望用于低算力场景。
HN 关注点：这种完全不同于梯度下降的建模范式能否与现有方法竞争，讨论焦点是基准结果偏弱、数学门槛高，以及它与决策树等经典方法的联系。
HN 评论精选
这几条评论里：支持 1 质疑 2 补充 1
质疑实测 sklearn 基线 MLP 在相同设置下成绩不低于论文结果，基准偏弱
我用 sklearn 的 MLPClassifier 在同样的 MNIST 数据划分上做了对比：10×20 样本得到 0.645，10×200 样本得到 0.838，而论文分别是 62% 和 76%。当然验证方式不同（我用全部剩余数据验证，他们做 20 次 70%/30% 划分，白白减少了训练样本；更公平的对比是 1400 样本时 0.819），但成绩至少相当。方法挺酷的，可以不纠结基准来看它——不过 Iris 和 Wine 现在实在是玩具数据集了。
princeofwands查看英文原文
from sklearn.datasets import fetch_openml from sklearn.model_selection import train_test_split from sklearn.neural_network import MLPClassifier from sklearn.metrics import accuracy_score X, y = fetch_openml('mnist_784', version=1, return_X_y=True) X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=1729, test_size=X.shape[0] - (10 * 20)) model = MLPClassifier(random_state=1729) model.fit(X_train, y_train) p = model.predict(X_test) print(accuracy_score(y_test, p)) X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=1729, test_size=X.shape[0] - (10 * 200)) model = MLPClassifier(random_state=1729) model.fit(X_train, y_train) p = model.predict(X_test) print(accuracy_score(y_test, p)) This gets you 0.645 and 0.838 accuracy score respectively (versus 62% and 76% in the paper). Sure, different validation (I validate on all the remaining data, they do 20x 70% 30% splits on the 200 and 2000 samples, which needlessly lowers the number of training samples, fairer comparison is 0.819 with 1400 samples), but the scores seem at least comparable. Cool method though, I can dig this and look beyond benchmarks (Though Iris and Wine are really toy datasets by now.)
补充该方法本质上是决策树嵌入的变体，应借鉴决策树领域已有研究
他们的超立方体覆盖形式化可以看作一种带特定划分规则的决策树归纳，只在标签统一的叶子处终止分支，但他们把树节点当作一种嵌入来套 softmax。我喜欢 ReLU 与几何表示之间的联系，用空间方式思考更容易。读的时候让我不断想起研究生时的经典机器学习课程。我喜欢这个方向，但如果承认这是决策树嵌入的变体、并建立在决策树领域海量研究工作之上会更好。我猜做随机森林版本会更有用，也许这能成为一个正经的预训练步骤。
tdj查看英文原文
Their hypercube covering formalism can be seen as decision tree induction with a specific partitioning rule, and terminating branching only at uniformly labeled leaves. But try are using the tree nodes as kind of an embedding to apply a softmax on. I like the connection between relus and the geometrical representation, makes it easier to think about in spatial terms. Reading this I got several dejavus to my grad school classes on classical ML stuff. I like the direction but it feels like it could be better if it admitted that it's a variant of decision tree embedding, and built on some of the massive amount of research work in that area. At least in terms of understanding. I suspect doing a random forest version of this would actually help. Perhaps we will see this as a legit pre-training step.
支持看好其在低算力场景的潜力，但数学门槛高可能阻碍后续研究
这个思路很有意思，数字也算有希望。既然它与传统建模方式完全不同，我认为只要达到基础浅层模型的水平就可以接受。如果这种构造式方法经过打磨后能与当前最先进模型相当，对 IoT 设备等低算力应用会非常有价值。说实话，尽管我有 ML/NLP 背景，数学部分我也只能大致跟上——其他 ML 研究者能看懂多少？我的印象是这些数学远超 ML 从业者通常掌握的水平。如果这种几何构造真能与迭代训练竞争，其复杂性可能会拖慢进展，但我希望这篇论文能吸引具备相应数学功底的人继续探索。
rococode查看英文原文
This seems like a really interesting approach and I think the numbers are promising. Given that it's an entirely different type of model-building than traditional methods, I think it's fine for it to just be up to par with a basic shallow model. If the constructive approach turns out to be comparable to current state-of-the-art models with sufficient refinement, it could be really valuable for low-compute applications like IoT devices, etc. To be honest, I can't say I know enough about the math here to do anything more than vaguely follow their explanations despite my ML/NLP background. I'm curious - other ML researchers out there, how much of this are you able to understand? My impression is that this math is pretty far beyond what ML folks typically know, although I'm on the lower end of the spectrum as far as math knowledge goes, so I may be totally wrong (and need to spend more time reading textbooks haha). I wonder if the complexity may slow down progress if it does turn out that this kind of geometric construction can compete with iterative training. It sounds like this approach could potentially support more complex networks by working more on the geometric representation, so I certainly hope this paper serves its purpose of motivating people with the right skillsets to do further exploration.
质疑质疑 76% 的 MNIST 准确率太低，应与 SVM 等简单基线对比
想知道这和单次训练的 SVM 或最近邻相比如何。MNIST 上 76% 说实话很尴尬。
tbenst查看英文原文
Wonder how this compares to single shot with an SVM or nearest neighbor. 76% on MNIST is frankly embarrassing
历次讨论
- 2019Hacker News 讨论帖148 分 · 17 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选