AI 与机器学习2024arxiv.org
嵌入的 Cosine-Similarity 真的关乎相似性吗？
Is Cosine-Similarity of Embeddings Really About Similarity?
进阶 · 约 30 分钟读完前置知识：词嵌入基础、线性代数与向量空间、相似度度量
导读
这篇论文从标题看是在追问：嵌入向量的余弦相似度是否真的等于语义相似性，以及把它当作相似性度量是否可靠。评论中普遍指出，余弦相似度对否定、反义词以及“等价还是相关”并不敏感，并进一步讨论嵌入空间应采用何种几何/距离度量。
HN 关注点：是因为 cosine similarity 被广泛用于判断 embedding 是否相似，而评论区围绕其语义盲区、反义词邻近和度量几何展开争论。
HN 评论精选
这几条评论里：质疑 1 补充 3
质疑余弦相似度无法区分否定造成的语义反转，可能高估结构相似但语义相反的文本。
没读论文，但余弦相似度对句子几乎没有语义理解。例如三元组：1“Yes, this is a demonstration”；2“Yes, this isn't a demonstration”；3“Here is an example”。<1,2> 的余弦相似度高于 <1,3>：前者结构上只差一个词，但语义在目标上相反；后者实际意思相同。如果论文是想说明余弦相似度在语义理解上的效力，那早已众所周知；如果讨论向量及底层运算，我会感兴趣。
latency-guy2查看英文原文
Not reading the paper, cosine similarity has little to no semantic understanding of sentences. E.g. the following triple 1: "Yes, this is a demonstration" 2: "Yes, this isn't a demonstration" 3: "Here is an example" <1, 2>, Has "higher" cosine similarity than <1, 3>, structurally equivalent except for one token/word, <1, 2> semantically means the opposite of each other depending on what you're targeting in that sentence. While <1, 3> means effectively the same thing. If this paper is about persuading people about efficacy with regards to semantic understanding, OK, but that was always known. If its about something with relation to vectors and the underlying operations, then I'll be interested.
补充嵌入相似度捕捉的是上下文可替换性，会混淆反义、等价与相关关系。
Embeddings 来自计算什么词能出现在给定上下文中，因此会出现在同一位置的词之间 cosine 分数更高。但它不进一步区分，所以“beautiful”和“ugly”可能嵌入得很近，尽管它们是反义词，因为它们常出现在相似位置。另一个局限是，嵌入和余弦相似度无法告诉你“有多相似”——是等价还是仅相关？它们把等价、反义和相关事物混在一起。
visarga查看英文原文
Embeddings result from computing what word can appear in a given context, so words that would appear in the same spot will have higher cosine score between themselves. But it doesn't differentiate further, so you can have "beautiful" and "ugly" embed very close to each other even though they are opposites - they tend to appear in similar places. Another limitation of embeddings and cosine-similarity is that that they can't tell you "how similar" - is it equivalence or just relatedness? They make a mess of equivalent, antonymous and related things.
补充质疑 cosine，强调嵌入差向量/方向而非绝对余弦才是有意义的操作。
为什么会有人认为 cosine 相似度是有用指标？现实中，物体在宇宙中的任意绝对位置（如果能测量）并不重要，最重要的是方向和到附近物体的距离。据我理解，两个词嵌入之差给出一个方向，神奇之处在于用这些方向得到新词，常引用的例子是 King-Man+Woman=Queen。这种观点何时失宠了？
mikewarot查看英文原文
Why would anyone expect cosine-similarity to be a useful metric? In the real word, the arbitrary absolute position of an object in the universe (if it could be measured) isn't that important, it's the directions and distances to nearby objects that matters most. It's my understanding that the delta between two word embeddings, gives a direction, and the magic is from using those directions to get to new words. The oft cited example is King-Man+Woman = Queen [1] When did this view fall from favor? [1] https://www.technologyreview.com/2015/09/17/166211/king-man-woman-queen-the-marvelous-mathematics-of-computational-linguistics/
补充相似度度量依赖隐含的几何结构，普通点积可能忽略流形局部曲率而不合适。
距离度量只和它（隐式）实现的伪黎曼度量一样好。若相信流形假设，这些度量应是局部的，因为流形曲率是局部性质。用普通点积比较地图上的直线是错的，因为那些线并非真正直线，没有考虑曲率张量中的丰富信息。用错内积类似“地平说”谬误。
cfgauss2718查看英文原文
Distance measures are only as good as the Pseudo-Riemannian metric they (implicitly)implement. If the manifold hypothesis is believed, then these metrics should be local because the manifold curvature is a local property. You would be mistaken to use an ordinary dot product to compare straight lines on a map of the globe, because those lines aren’t actually straight - they do not account for the rich information in the curvature tensor. Using the wrong inner product is akin to the flat-Earth fallacy.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2024Hacker News 讨论帖210 分 · 115 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选