神经科学与心理学2018colala.bcs.rochester.edu
一个参数总是够用
One parameter is always enough
进阶 · 约 25 分钟读完前置知识：函数逼近与拟合基础、实数与无限精度、模型选择基础
导读
这篇论文讨论模型参数数量与拟合能力的关系，主张在参数可取无限精度实数的前提下，单个参数就足以表示或拟合任意复杂的数据/函数，并给出一种闭式构造。核心争议在于：这种构造把大量信息编码进一个无限精度的实数，实际建模中的参数约束和不确定性并未因此消失。
HN 关注点：这个结论是否只是“无限精度实数可编码任意信息”的数学把戏，以及它是否忽视参数不确定性和模型选择的实际意义。
HN 评论精选
这几条评论里：质疑 2 补充 2
质疑结论依赖无限精度参数，实际中 theta 无法被约束，模型选择意义被夸大。
他们的方程很可爱，但这并不令人惊讶，意义也没有他们暗示的那么大。结果严重依赖 theta 参数具有无限精度。一个无限精度的实数可以编码任意多信息。单精度 float 用 4 字节，double 用 8 字节；若只需单精度，可以把两个 float 塞进一个 double。把 double 换成需要无限字节表示的无限精度数，就能塞进任意多个有限精度 float。他们基本就是这么做的，只是给出了简单的闭式解码表达式。意义被夸大，是因为模型极其混沌地依赖 theta。论文里的图需要模型参数有几十万位精度。评估模型不能只看拟合好坏和参数个数，还要看参数约束程度和拟合的不确定带。这个模型里 theta 完全不受约束，不确定带会覆盖整个数据范围。
foob查看英文原文
Their equation is cute, but this really isn't remotely surprising and the implications aren't as significant as they imply. The result relies heavily on the fact that their theta parameter has infinite precision. You can encode as much information as you want in a single real number with infinite precision. Think of it this way: a single precision float requires 4 bytes to store while a double requires 8. If all you need is single precision, then you can store two floats inside of one double variable with each occupying 4 of the 8 bytes. Now replace the double with an infinite precision number that takes infinite bytes to represent. Once you have an infinite number of bytes to work with, you can pack in as many floats of finite precision in there as you want. That's basically what they're doing here, they just have a simple closed form expression for decoding it. The reason that the implications are a bit overblown is that their model is tremendously and chaotically dependent on the value of theta. The plots that they include in the paper require hundreds of thousands of digits of precision in the model parameter. Nobody evaluates a model based simply on how well it fits the data and its number of parameters; you also look at how well the model parameters are constrained and what the uncertainty bands on the fit are. With this model, theta would be completely unconstrained and their uncertainty bands would cover the entire range of the data that they're fitting. It simply doesn't matter how many parameters you have when that's the case, it means that your fit is useless.
补充单实数编码可计算函数是已知的数学事实，并不新鲜。
当然，一个精度任意大的参数就够了。85 多年前哥德尔已经表明，每个可计算函数无论多复杂，都可表示为单个实数。这一思想用在他证明哥德尔定理的对角化步骤中。
todd8查看英文原文
Of course one arbitrarily large precision parameter is enough. Over 85 years ago Gödel demonstrated that every computable function, no matter how complex, has a representation as a single real number. This idea was used in the diagonalization step in his proof of Gödel's Theorem.
补充这本质是数据编码/渲染，类似 Tupper 自指公式和算术压缩。
是的，把数据编码进数字，再构造一个本质上相当于位图渲染器的方程。这基本上就是计算机一直在做的事，只是我们通常没意识到，因为大多数时候它们不这样展示数据。这让我想起 Tupper 的自指公式，也和算术压缩的工作原理有关。
userbinator查看英文原文
Yes, encoding data into numbers and creating an equation that essentially provides a bitmap renderer. Basically what computers do all the time, but we just don't realise it because most of the time they don't show the data that way. Reminds me of https://en.wikipedia.org/wiki/Tupper%27s_self-referential_formula Also related to how arithmetic compression works.
质疑数学上显然，但自然参数数量对理解现实很重要。
很容易把 R 嵌入 R^N 且像集稠密，所以单参数总是足够这一点很直接。对于可计算实数（数值计算中处理的一切），这个嵌入甚至是双射。但这完全没抓住重点，因为“自然的”参数数量对理解现实很重要。
madez查看英文原文
One can easily embed R into R^N with a dense image, so it's straightforward to see that one parameter is always enough. In the case of computable reals (which contain everything we deal with when doing things numerically), the embedding is even bijective. This completely misses the point, though, because the "natural" number of parameters is important in understanding reality.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
- 评论区提到的论文
历次讨论
- 2018Hacker News 讨论帖143 分 · 46 评
同领域还值得读
神经科学家能理解微处理器吗？（2017）
Could a Neuroscientist Understand a Microprocessor? (2017)
提出一个思想实验：把神经科学常用的分析方法（损伤实验、电生理、功能定位等）应用到一个我们完全了解其原理的工程系统——6502 微处理器（运行游戏）上，检验这些方法能否真正“理解”…
神经科学与心理学2017 · 411 分 · 135 评进阶 · 约 60 分钟编辑精选大脑衰老呈现非线性转变，提示中年“关键窗口”
Brain aging shows nonlinear transitions, suggesting a midlife "critical window"
这篇 PNAS 论文研究大脑衰老的非线性变化，发现大脑老化并非匀速进行，而是在中年（约 40-60 岁）存在一个关键的"临界窗口期"，此时干预可能有效。
神经科学与心理学2025 · 295 分 · 159 评进阶 · 约 75 分钟编辑精选社交网络中大规模情绪传染的证据（2014）
Evidence of massive-scale emotional contagion through social networks (2014)
这篇 2014 年发表于 PNAS 的论文研究了 Facebook 上约 70 万用户的信息流中，减少正面或负面内容是否会传染性地影响用户自己发布内容的情绪。
神经科学与心理学2020 · 285 分 · 72 评入门 · 约 45 分钟编辑精选裸盖菇素使人类大脑去同步化
Psilocybin desynchronizes the human brain
这篇 Nature 研究用个体化的 fMRI 精密成像追踪受试者服用裸盖菇素（psilocybin）前后的脑功能连接变化，发现药物会造成大范围的脑网络「去同步化」，尤其是默认模式…
神经科学与心理学2024 · 273 分 · 259 评进阶 · 约 60 分钟编辑精选对智力的赞扬会削弱儿童的动机与表现（1998）
Praise for Intelligence Can Undermine Children's Motivation (1998)
这篇 1998 年的经典论文（Dweck 等人）通过实验研究表扬方式对儿童动机的影响：夸孩子“聪明”（表扬智力）会让他们回避挑战、害怕失败，而表扬努力则更能培养坚持性。
神经科学与心理学2016 · 253 分 · 155 评入门 · 约 45 分钟编辑精选