AI 与机器学习2025arxiv.org
在动态视频环境中评测视觉语言模型的 OCR 能力
Benchmarking vision-language models on OCR in dynamic video environments
进阶 · 约 45 分钟读完前置知识：视觉语言模型基础、OCR 基本原理、基准评测方法
导读
这篇论文构建了一个动态视频环境下的 OCR 基准，用于评测视觉语言模型（Claude-3、Gemini-1.5、GPT-4o）在视频帧中识别文字的能力，并与传统 OCR 工具（EasyOCR、RapidOCR）对比。结论是各模型在动态场景下的文字识别仍存在明显不足。
HN 关注点：VLM 在真实动态场景下 OCR 的实际能力，讨论焦点集中在基准的 ground truth 质量和'最先进模型'选择是否过时。
HN 评论精选
这几条评论里：质疑 3 亲历 1
质疑基准对比对象不公平，且未说明 temperature 设置，结论可信度存疑
这看起来像是'和一群落后十年的人（非 transformer、纯视觉模型）以及没认真优化 OCR 的模型比，Google 表现很好'。EasyOCR 是 2007 年的 LSTM-CTC，RapidOCR 是 2021 年的 ConvNet 方案，都主打速度，在速度和内存上远胜任何 transformer 模型，但精度本就不是最先进的——这是众所周知的。另外 GPT-4o 的结果看起来更合理：'COCONUT' vs Gemini 的 'CONU CNBC'，ground truth 是 'C CONU CNBC'，而人类显然会答 'COCONUT MILK'。GPT-4o 表现更好可能与 temperature 设置有关，但论文完全没提是否调整了 temperature，这是个很大的疏漏。
spwa4查看英文原文
This looks a lot like "compared to a bunch of people who are 10 years behind (non-transformer, vision-only models), and people who aren't trying (aren't optimizing for OCR) Google is doing real well" EasyOCR is LSTM-CTC from 2007, RapidOCR is a ConvNet approach from 2021, both focused on speed. Both will vastly outperform almost any transformer model, and certainly a big one, on speed and memory usage, but they aren't state of the art on accuracy. This is well known, for a decade at this point. 2 decades for LSTM-CTC. Plus, I must say the GPT-4o results look a lot saner. "COCONUT" (GPT-4o) vs "CONU CNBC" (Gemini) vs Ground Truth "C CONU CNBC". And, obviously the ground truth should be "COCONUT MILK" (the word milk is almost entirely out of the picture, but is still the right answer that a human would give). The "C CONU" comes from the first O of COCONUT being somewhat obscured by a drawing of ... I don't know what the hell that is. It's still very obvious it's meant to be "COCONUT MILK", so the GPT-4o answer is still not quite perfect, but heaps better than all the others. Now this looks very much like it might be temperature related, and I can find nothing in the paper about changing the temperature, which is imho a very big gap (temperature gives transformer models more freedom to choose more creative answers. The better performance of GPT-4o might well be the result of such a more creative choice, and might also explain why Gemini is trying so hard to stay so very close to the ground truth. It's still quite the accomplishment to succeed, but GPT-4o is still better)
质疑ground truth 本身有错，作者盲目跟随标注导致误判模型
软件有时比所谓的 ground truth 做得更好（是谁标注的？），我不明白作者为何盲目标注答案，审稿人居然也通过了。图 1 中作者抱怨 Gemini 把 'ss ety!' 误读为 'ness ety!'，但随便看一眼图就知道 Gemini 读的是对的。图 11 中说 Claude '打乱了 ground truth 的自然语序'，可 ground truth 的顺序根本不通，Claude 的顺序反而合理。
alberto-m查看英文原文
It seems to me that the software is occasionally doing better than the supposed “ground truth” (who annotated that?), and I don't understand why the authors are blindly following the latter, and the reviewers apparently approved that. In Figure 1 the authors complain that Gemini “misreads 'ss ety!' as 'ness ety!'”, but even a casual look at the image reveals that Gemini's reading is correct. In Figure 11, they state that Claude is “altering the natural sequence of ideas in the ground truth”, except that the sequence in the ground truth makes no sense, while Claude's order does (only the initial “the” is misplaced).
质疑批评论文把已过时的模型称为'最先进'，反映学术界跟不上 AI 迭代速度
论文摘要声称 Claude-3、Gemini-1.5、GPT-4o 是'三个最先进的 VLM'，但它们都不是最先进的。学术界完全跟不上 AI 发展速度，这在研究论文中极其常见。摘要第一句就错得离谱，我为什么要读剩下的部分？
alberto-m查看英文原文
It seems to me that the software is occasionally doing better than the supposed “ground truth” (who annotated that?), and I don't understand why the authors are blindly following the latter, and the reviewers apparently approved that. In Figure 1 the authors complain that Gemini “misreads 'ss ety!' as 'ness ety!'”, but even a casual look at the image reveals that Gemini's reading is correct. In Figure 11, they state that Claude is “altering the natural sequence of ideas in the ground truth”, except that the sequence in the ground truth makes no sense, while Claude's order does (only the initial “the” is misplaced).
亲历实际使用中现有 AI 工具处理混合文档查询仍不实用，痛点未解决
我有很多客户文件，试遍了各种 AI 工具（付费或自托管），想实现指向一个含 xlsx 和 pdf 的文件夹就能问'M Smith 合同的截止日期是什么'或'M Smith 还欠多少钱'，但都很失望：要么很复杂，要么遇到非文本 PDF 就崩。如果需要提供 schema、预处理数据，那 AI 提供的不过是自然语言 SQL，对技术用户来说并没有消除真正的痛点。也许是我没找对工具？有人知道真正能用的方案吗？
nolok查看英文原文
I have lots of customer files and I've looked around with all these AI tools for something, paid or self hosted or whatever, where I point it to a folder with xlsx and pdf and then I can query "Whats the end date or M Smith contract" or "How much does M Smith still owe" and I've been very disappointed by that, it's either very complicated, or they break down with non text based pdf, or... It feels to me that if you need to provide schema and preprocess the data and this and that at the end all AI provide is a way to do some SQL in natural language, meaning yes it's better but it doesn't remove the actual pain point if you're a tech user. Then again maybe I'm wrong, didn't find the right tool or didn't understand it. Is what I'm looking for something that actually exists (and works, not just on simple cases)?
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2025Hacker News 讨论帖142 分 · 58 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选