经济与社会科学2021pnas.org
大型科学领域中经典进展的放缓
Slowed canonical progress in large fields of science
进阶 · 约 50 分钟读完前置知识：科学计量学与引文分析、科研激励机制、统计与网络数据分析基础
导读
这篇发表在 PNAS 上的论文研究的是：在规模较大的学科里，新论文取代旧经典论文（即“经典更替”）的速度为何变慢。作者用引用数据来衡量学科进展，发现领域规模扩大后，研究者更倾向于围绕已有热点选题，导致新论文更难撼动既有经典，从而拖慢整体知识更新。
HN 关注点：这一结论如何与科研激励、A/B 测试式的指标化考核以及“可测量才被奖励”的普遍问题相互印证。
HN 评论精选
这几条评论里：质疑 1 补充 2 亲历 1
亲历指标化和“客观化”会激励人们追逐容易测量的短期结果，使组织困在局部最优。
我也在极度“数据驱动”、以 A/B 测试效果论成败的组织里发现了类似问题。问题不在于 A/B 测试背后的科学，而在于个人被激励去做那些容易测量的事；需要很久才见效、或初期有破坏性但长期有益的事情则被低估。这个主题在商业、体育、政治、学术界等很多领域都存在：当我们拥有大量数据并渴望尽可能“客观”时，就很容易被困在同质的“局部最优”里，因为我们只用最容易测量的东西来评分。
hn_throwaway_99查看英文原文
I've also found very similar issues with extremely "data driven" organizations that live and die by A/B test performance. It's not that there is anything fundamentally wrong with the science behind A/B testing, it's just that individuals are incentivized to run tests on things that are easily measured. Things where results will take a long time to show, or things that may be initially disruptive but then beneficial, are discounted. I see this theme in many, many areas: business, sports, politics, academia, etc. When we have tons of data and a desire to make things as "objective" as possible, it's easy to get stuck in homogeneous "local maxima" because we just grade by the things that are easiest to measure.
补充补充文献：科研激励结构会“自然选择”出坏方法，只有制度改革才能扭转。
Smaldino 与 McElreath（2016）的《坏科学的自然选择》：糟糕的研究设计与数据分析会助长假阳性结果；这类糟糕方法屡禁不止，说明问题不只是理解偏差，而是激励在偏袒它们，于是“坏科学”被自然选择出来。有些规范的分析方法几乎可以肯定是为了更容易发表，而不是为了发现。要改进科学文化，必须从纠正误解转向奖励真正的理解。作者随后给出一个科研社群动态模型：相互竞争的实验室用文化传播的研究方法检验新假设或已发表的假设；与现实一样，成功的实验室产生更多“后代”，其方法被更多复制，学生也更可能自立门户。以高产出为导向的选择导致方法变差、假发现率升高；重复实验能减缓但无法阻止方法退化。提升研究质量需要制度层面的变革。
shoo查看英文原文
Smaldino & McElreath (2016) -- The natural selection of bad science https://royalsocietypublishing.org/doi/pdf/10.1098/rsos.160384 > Poor research design and data analysis encourage false-positive findings. Such poor methods persist despite perennial calls for improvement, suggesting that they result from something more than just misunderstanding. The persistence of poor methods results partly from incentives that favour them, leading to the natural selection of bad science. [...] Some normative methods of analysis have almost certainly been selected to further publication instead of discovery. In order to improve the culture of science, a shift must be made away from correcting misunderstandings and towards rewarding understanding. [...] To demonstrate the logical consequences of structural incentives, we then present a dynamic model of scientific communities in which competing laboratories investigate novel or previously published hypotheses using culturally transmitted research methods. As in the real world, successful labs produce more ‘progeny,’ such that their methods are more often copied and their students are more likely to start labs of their own. Selection for high output leads to poorer methods and increasingly high false discovery rates. We additionally show that replication slows but does not stop the process of methodological deterioration. Improving the quality of research requires change at the institutional level.
质疑质疑用引用更替衡量科学进展，认为想法是集体累积的，功劳归属机制已改变。
我本来很期待读这篇，但细想结果后有些失望：进展真的能用引用更替来衡量吗？这让我想到“历史伟人”史观。最近读到 LTP 发现者 T. Lomo 的回顾，他说很多人问他为什么原始论文没引 Hebb（通常被视为可塑性理论的开创者）；他回答说，Hebb 只是写下了大家已知的东西，因此领域内没人觉得值得引用。可如今神经科学和机器学习的人经常引 Hebb，却很少引 Lomo 和 Bliss——那是否说明投给 Hebb 的科研经费花得值，而支持那些实验的经费不值？另一种进展观认为，想法像吸积形成行星：一堆看似不起眼的论文聚拢，然后有人说“等等，这里有颗行星！”但行星不是他创造的，许多单个“岩石”其实早就明白发生了什么。过去守门人允许“这是行星”的论文获得影响力，如今 50 个人说同样的话，却没人赢得功劳。
ckemere查看英文原文
I was excited to read this, but was disappointed upon thinking further about the results. Is progress is measured by turnover in citations? It strikes me that the idea here is linked to the great-person concepts in history. It reminds me of a retrospective I recently read by T. Lomo, who is widely credited with the discovery of LTP. He said that many people ask why his original paper doesn't cite Hebb, who is usually described as the originator of plasticity theory. His response is that Hebb wrote down what everyone already knew, and that no one in the field found it worth citing for that reason. Of course modern neuroscience and machine learning people cite Hebbs work regularly but not Lomo and Bliss. Should we then infer that the research dollars given to Hebb were well spent and those that supported the experiments were not? An alternative view of progress would say that ideas are like accretion turning into a planet - a bunch of seemingly insignificant papers gravitate together and then someone says - "wait, there's a planet here!" But they didn't create the planet, and probably many of the individual rocks already understood exactly what was going on. But in the old days the gate keepers allowed the "this is a planet!" paper to gain influence/significance, but now 50 different people say the same thing but none of them win the credit battle...
补充补充阅读：科学严谨性危机与同行评审的封闭性。
推荐一本书：《Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions》。总体而言，所有科学领域都存在由同行构成的“围墙花园”及其带来的严重问题。另外推荐 Eric 与 Brett Weinstein 的一场讨论，它只呈现了这些大问题中的一个例子。
lettergram查看英文原文
I recommend the book: Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions Generally, all scientific fields have immense issues related to the walled gardens and limitations created by their peers. I also recommend this discussion between Eric and Brett Weinstein which highlights just one example of the major issues: https://m.youtube.com/watch?v=JLb5hZLw44s
历次讨论
- 2021Hacker News 讨论帖178 分 · 72 评
同领域还值得读
一切的回报率，1870–2015
The Rate of Return on Everything, 1870–2015
利用1870–2015年多国的长期数据，估算股票、住房等各类资本的回报率，并将其与经济增长率进行比较。
经济与社会科学2019 · 487 分 · 260 评进阶 · 约 100 分钟编辑精选一切资产的回报率，1870–2015
The Rate of Return on Everything, 1870–2015
构建了 1870–2015 年 16 个国家的长周期资产回报数据集，覆盖股票、住房、政府债券和短期票据，系统比较各类资产的真实回报与波动性。
经济与社会科学2018 · 215 分 · 151 评进阶 · 约 120 分钟编辑精选一切资产的回报率，1870–2015（2019）
The Rate of Return on Everything, 1870–2015 (2019)
这篇发表于《经济学季刊》的论文构建了 1870–2015 年间 16 个主要发达国家的股票、债券、房地产等资产收益率的长周期数据库。
经济与社会科学2024 · 190 分 · 148 评进阶 · 约 80 分钟编辑精选想法是否越来越难找？
Are ideas getting harder to find?
探讨“想法是否越来越难找到”：通过衡量多个领域的“研究生产率”，发现维持相同的增长需要投入指数级增长的研究资源，即研究效率在持续下降。
经济与社会科学2016 · 182 分 · 106 评入门 · 约 60 分钟编辑精选比特币与区块链的经济极限
The Economic Limits of Bitcoin and the Blockchain
芝加哥大学 Eric Budish 的论文从经济学角度分析比特币与区块链的根本限制：核心论点是只要攻击网络（双花或砸盘做空）的收益超过挖矿成本，51% 算力攻击就有利可图，而挖矿…
经济与社会科学2018 · 176 分 · 37 评进阶 · 约 90 分钟编辑精选