AI 与机器学习2025arxiv.org
使推理模型自我提升的认知行为
Cognitive Behaviors That Enable Self-Improving Reasoners
进阶 · 约 90 分钟读完前置知识：大语言模型基础、RLHF/强化学习训练流程、DeepSeek R1 与推理模型背景
导读
论文研究语言模型为何能具备自我改进的推理能力，聚焦验证、回溯、子目标设定和反向链接这几类认知行为，并指出专家人类解题者与表现较好的语言模型都会使用它们。它可能通过分析这些行为与推理成功之间的关系，探讨如何训练或激发模型的自我改进推理。
HN 关注点：把 AI 推理与人类解题策略联系起来，并由此引发“AI 训练方法能否反过来训练人类思维”以及自我改进模型内部过程不透明的讨论。
HN 评论精选
这几条评论里：质疑 1 补充 2 亲历 1
亲历亲历：模仿 R1 的显式推理过程可能提升人的解题表现，AI 认知行为或可反哺人类。
论文指出验证、回溯、子目标设定和反向链接这四种关键认知行为，既被专家人类解题者使用，也被成功的语言模型使用。随着我们改进 AI，也许也会无意中找到提升人类智能的方法。我最近备考时就有亲身体验：读练习题时大声说出推理过程，模仿 DeepSeek R1 的推理方式/“人格”。花很多时间读 R1 冗长的输出，相当于对我的大脑进行了推理任务的微调；我认为这帮助我考试取得高分。
owenpalmer查看英文原文
> four key cognitive behaviors -- verification, backtracking, subgoal setting, and backward chaining -- that both expert human problem solvers and successful language models employ. As we make AI better, perhaps we'll inadvertently find ways to make HI (human intelligence) better too. I had a personal experience with this when I was studying for an exam recently. As I read over practice questions, I spoke aloud, replicating the reasoning methods/personality of Deepseek R1. By spending a lot of time reading long verbose R1 outputs, I've essentially fine-tuned my brain for reasoning tasks. I believe this method contributed to my excellent score on that exam.
补充内部独白/chain-of-thought 与计划执行功能有关，用笔记外化推理可弥补。
我有时看到 Reddit 上讨论“内部独白”体验的帖子。我没有这种独白，至少我的“自我”无法访问它；但我常想这种独白是否类似 chain of thought。我觉得如果没有这种“想法流”，我的计划和执行功能可能不如某些人有效。做一点 chain-of-thought 笔记时，我确实在这些任务上更有效。我还怀疑我较少反复思虑和事后怀疑自己，也较少有焦虑行为——我猜如果整天有声音在耳边说话可能会有这些；不过这可能跑题。
idiotsecant查看英文原文
I sometimes see these reddit threads of people talking about the experience of having an internal monologue. I have no such monologue, at least not one that is accessible to the part of my mind that calls itself 'me', but I have often wondered if that monologue is something like a 'chain of thought'. I feel like maybe without access to that 'idea feed' maybe my planning and executive functioning is less effective than some other people. I do find myself quite more effective with those sort of tasks when I do a little 'chain of thought' notepad. I also suspect I spend less time ruminating and second-guessing myself and other anxious behaviours that I imagine would come with having someone talking in your ear all day, but that's probably off topic.
补充追问 AI 训练方法能否迁移到人类思维训练。
我们对 AI 训练技术的了解，在多大程度上帮助发现了如何更好地训练人们思考？
robocat查看英文原文
How much has our knowledge of AI training techniques helped to discover how to train people to think better?
质疑担忧自我改进 AI 转向人类不可解读的内部语言和多智能体交流，过程不透明。
确实，但问题是自我改进的 AI 会导致一种有点令人不安的思维方式。AI 会切换到一种内部胡言乱语式的语言，看起来毫无意义，但显然仍对 AI 传达意义，然后用这种语言思考（如果算语言的话，但不确定还能是什么），最后给出正确结果。更糟的是，当你用多个 agent 让 AI LLM 互相交谈时，所有 AI agent 都会切换到这种内部语言，尽管没有人理解到底发生了什么，它们仍在推进。这似乎非常糟糕。例如问 strawberry 里有几个 r：它会先输出一串人类看不懂的乱码式内容，最后给出“strawberry 里有 3 个 r”。
spwa4查看英文原文
True, but a problem is that self-improving AI leads to a somewhat troubling mode of thinking. AIs switch to an internal babbling type language that makes no sense but clearly still conveys meaning to the AIs, then think in that language (if it's a language, though not sure what else it could be) and then produce correct results. Worse, when you use multiple agents to get AI LLMs talking to one another, all AI agents switch to this internal language and they make progress despite no human understanding what hell is happening. This seems very bad. Illustration: > How many r in strawberry? I'm asked how many r in strawberry. I can just spell the word and a;dklsjaw; a;ewjraqwpeouypaads;lq qepwiouryaqeopw qewrpoiuyoiauysdqw145124rfa.nkjlwh ;45a8345a894ya4a q4p58q45jaq;lkjas;dlfkja;j <answer>There are 3 (three) r's in strawberry</answer>
历次讨论
- 2025Hacker News 讨论帖279 分 · 103 评
同领域还值得读
DeepSeek-R1：通过强化学习激励 LLM 的推理能力
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
研究如何用强化学习激励大语言模型的推理能力，提出 DeepSeek-R1，并探讨 RL 训练对模型推理表现的影响。
AI 与机器学习2025 · 1,351 分 · 1,056 评 · 引用 30进阶 · 约 80 分钟编辑精选1-bit LLM 时代：面向低成本计算的三值参数
The Era of 1-bit LLMs: ternary parameters for cost-effective computing
提出用三值参数 {-1, 0, 1} 替代 LLM 中的浮点权重，即所谓“1-bit LLM”。
AI 与机器学习2024 · 1,040 分 · 447 评 · 引用 19进阶 · 约 45 分钟编辑精选DeepSeek-V3.2：推进开放大语言模型的前沿
DeepSeek-v3.2: Pushing the frontier of open large language models
这是 DeepSeek-V3.2 的技术报告，延续了 DeepSeek 开源权重模型的路线，目标是让开放模型继续逼近甚至追平前沿闭源模型。
AI 与机器学习2025 · 982 分 · 465 评进阶 · 约 90 分钟编辑精选通过自我对弈与通用强化学习掌握国际象棋和将棋（AlphaZero）
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
DeepMind 的 AlphaZero 论文：用通用强化学习算法（自我博弈 + MCTS + 神经网络），从零开始仅数小时训练便在国际象棋和将棋上击败了世界顶级引擎 Stock…
AI 与机器学习2017 · 539 分 · 270 评 · 引用 13进阶 · 约 90 分钟编辑精选用强化学习发现更快的矩阵乘法算法
Discovering faster matrix multiplication algorithms with reinforcement learning
论文提出 AlphaTensor，用强化学习搜索矩阵乘法算法。
AI 与机器学习2022 · 518 分 · 111 评专业 · 约 90 分钟编辑精选