AI 与机器学习2016arxiv.org编辑精选
高效文本分类的若干技巧
Bag of Tricks for Efficient Text Classification
入门 · 约 45 分钟读完前置知识：文本分类基础、词向量基础、softmax 分类
导读
这篇论文（fastText）研究如何高效地进行文本分类：作者来自 Facebook AI Research，提出用线性模型结合 n-gram 特征、特征哈希和层级 softmax 等简单技巧，在保持极快训练速度的同时达到接近深度神经网络的分类精度。
HN 关注点：如此简单的线性方法竟能媲美深度模型，讨论焦点在于其与 Vowpal Wabbit 等既有工具的关系，以及评测设置是否合理。
后来怎么样
fastText 成为工业界广泛使用的文本分类与词向量工具，其开源库至今仍被大量生产系统采用，论文提出的简单技巧组合成为高效文本分类的标准做法。
HN 评论精选
这几条评论里：质疑 2 补充 1
补充论文方法与 Vowpal Wabbit 几乎等价，核心都是 n-gram + 特征哈希 + 层级 softmax + 浅层模型
对于想高效做文本分类的人，我强烈推荐 Vowpal Wabbit（VW），它速度极快，在生产和研究中都被广泛使用，开箱即有大量配置选项。有研究者指出，VW 加上几个命令行参数（vw --ngrams 2 --log_multi [K] --nn 10）几乎就等价于论文描述的系统。两者快的背后是相同的技术：n-gram 特征、特征哈希技巧（类似 Bloom filter）、层级 softmax（用平衡二叉树把查找从 O(n) 降到 O(log n)），以及用浅层模型代替深度模型。不过 Facebook 团队或许还会提供更深入的洞见，目前论文略显单薄，源码也还在发布中。
Smerity查看英文原文
For anyone who is interested in efficiently classifying text, I can't recommend Vowpal Wabbit[1] (VW) enough. It's blazingly fast and has been used in both production and research. It also has a billion options out of the box for various different set-ups. Other researchers have noted[2] that with a set of command line flags that vw is almost the same as the system described in the paper, specifically, "vw --ngrams 2 --log_multi [K] --nn 10". Behind the speed of both methods is use of ngrams^, the feature hashing trick (think Bloom filter except for features) that has been the basis of VW since it began, hierarchical softmax (think finding an item in O(log n) using a balanced binary tree instead of an O(n) array traversal) and using a shallow instead of deep model. I am still interested in the more detailed insights the team from Facebook AI Research may provide but the initial paper is a little light and they're still in the process of releasing the source code. ^ Illustrating ngrams: "the cat sat on the mat" => "the cat", "cat sat", "sat on", "on the", "the mat" - you lose complex positional and ordering information but for many text classification tasks that's fine. [1]: https://github.com/JohnLangford/vowpal_wabbit/wiki [2]: https://twitter.com/haldaume3/status/751208719145328640
质疑质疑用无序多分类方式处理有序的评分数据，忽视领域信息会误导实际应用
我不明白为什么在 Yelp/IMDB 数据集上做这个实验。这些评分明显是有序的，但论文完全忽略了这一点，好像 Yelp 上 4 星和 5 星是完全不同的类别。更有意义的评估应该是预测评分与真实评分的接近程度，比如用均方误差，并对完全评错的预测施加更大惩罚。我理解这些数据集只是用来对比算法，但为什么不换成主题分类这种真正的多分类问题？机器学习论文中普遍忽视领域特定信息，这会让实际应用变得困难——现实中评估指标不同，在这些基准上被标记为较差的分类器实际可能更好。
lqdc13查看英文原文
I don't understand why they're doing this on the Yelp/IMDB datasets. Here's the paper they're citing that is doing the same thing with gated neural nets: http://www.emnlp2015.org/proceedings/EMNLP/pdf/EMNLP167.pdf However that paper also has the polar score which is a 2-class classifier - 1 star + 2 star vs 3 star + 4 star. Basically, the rating scores are clearly ordinal, but they're completely disregarding that information as if a 4 is a completely different star review than a 5 in a yelp restaurant review or a 9/10 is different from 8/10 for IMDB review. More useful estimate of success would be how close they got to real rating and possibly penalize disproportionately when a model gets the rating completely wrong. Something like mean squared error of the final score vs predicted score. I understand that these are just used to benchmark the algorithms against each other, but why not use something more relevant like a topic classification? That is a real n-classes problem. General disregard for domain-specific information is very prevalent in machine learning papers and it makes real world applications difficult because real world evaluation metrics are different so classifiers that are marked as inferior in such evaluation might actually be better. In both of their evaluations, why not try F(y) = exp(yj) / sum(exp(yi) for i in range(1, n+1)) p1 = F(y1) pj = F(yj) - F(yj-1), for j ≥ 2 for softmax? In other words, subtract cum prob of previous category. Or use kappa function? At least the naive bag of words comparison classifier should have used ordinal logistic regression instead of n-class logistic classifier.
质疑疑问：非循环网络如何处理词序列输入
非循环（非递归）网络是如何接收一个词序列输入的？
kwrobel查看英文原文
How a sequence of words is feeded to non-recurrent network?
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2016Hacker News 讨论帖169 分 · 15 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选