其他2016osf.io
29 个团队使用同一数据集，得出相互矛盾的结果
Twenty-nine teams use same dataset, find contradicting results
入门 · 约 45 分钟读完前置知识：统计学基础、假设检验、科学方法论
导读
29 个团队使用同一数据集独立分析，却得出了相互矛盾的结论。这项众包式研究将“分析方法的差异”本身作为实验对象，检验分析决策对结果的影响；不过从置信区间看，多数团队的结果区间仍有重叠。
HN 关注点：科学可重复性问题：同样的数据、不同的分析选择竟能导出相反结论，这直接挑战了单一分析结果的可信度。
HN 评论精选
这几条评论里：支持 1 补充 3
支持把不同分析方法当作多次“实验”来检验结论稳健性，是应对可重复性危机的好思路
这篇论文的妙处在于把分析方法本身透明地纳入了实验。可重复做对照实验的研究不太容易因分析技术不同而结果各异；但受伦理或现实限制的研究（多数涉及人类的实验）则高度依赖分析阶段的决策。这篇论文通过让不同团队尝试不同分析方法、相当于各自做一次“新实验”，来看是否达成共识。有意思的是，只有 2 个特征被超过 50% 的团队使用，说明无论方法如何，结果依然成立。类似的开放数据+分布式分析方式，是解决科学界可重复性危机的好办法。
entee查看英文原文
This paper is awesome because it transparently folds the analytical approach into the experiment being conducted. There are two kinds of scientific study: those where you can run another (ideally orthogonally approaching to the same question) experiment along with rigorous controls, and those where you can't. The first type is much less likely to have results vary based on analytical technique (effectively the second experiment is a new analytical technique). Of course it does happen sometimes and sometimes the studies are wrong, still more controls and more experiments are always more better. However, studies were you're limited by ethical or practical constraints (i.e. most experiments involving humans) don't have that luxury and therefore are far more contingent on decisions made at the analysis stage. What's awesome with this paper is it kind of gets around this limitation by trying different analytical methods, effectively each being a new "experiment" and seeing if they all reach the same consensus. Interestingly, very few features in the analysis were shared among a large fraction of the teams, (only 2 features were used by more than 50% of teams) which suggests that no matter the method, the result holds true. A similar approach to open data and distributed analysis would be a really great way to eliminate some of the recent trouble with reproducibility in the broader scientific literature.
补充补充资料：Nature 博文介绍研究背景
Nature 上有一篇介绍这项研究背景的博文：http://www.nature.com/news/crowdsourced-research-many-hands-make-tight-work-1.18508
dang查看英文原文
A blog post giving background is at http://www.nature.com/news/crowdsourced-research-many-hands-make-tight-work-1.18508.
补充建议用“反向结论论文”盲审机制来检验研究结论
这让我想到 Robin Hanson 的想法：在同行评审时增加一层盲审——用原始数据另写一篇结论相反的论文，随机分配给审稿人，只有当审稿人拒绝那篇反向论文时，原论文才能被接收。
SilasX查看英文原文
Reminds me of the idea (Robin Hanson's, I think?) to add an extra layer of blindness to studies: during peer review, take the original data, and write a separate paper with the opposite conclusion. Randomize which reviewers get which version. Your original paper is then only accepted if they reject the inverted version.
补充补充资料：FiveThirtyEight 报道；且各团队置信区间大多重叠
FiveThirtyEight 也写过这篇论文的报道（第二部分）：http://fivethirtyeight.com/features/science-isnt-broken/。好的一面是，看这 29 项研究的 95% 置信区间，几乎全部都有重叠。
sndean查看英文原文
FiveThirtyEight did a write up of this paper (part 2): http://fivethirtyeight.com/features/science-isnt-broken/ On the bright side, if you look at the 95CI for the 29 studies, almost all of them overlap.
历次讨论
- 2016Hacker News 讨论帖172 分 · 32 评
同领域还值得读
每个好的系统调节器都必须是该系统的一个模型 (1970)
Every Good Regulator of a System Must Be a Model of That System (1970)
这是控制论经典论文（Conant 与 Ashby，1970）：它试图论证，任何“好的”调节器都必须是其所调节系统的模型——即调节器与被控系统之间必须存在同构或模型关系。
其他2018 · 202 分 · 59 评进阶 · 约 45 分钟编辑精选复杂系统如何失效
How complex systems fail (2002)
总结了复杂系统如何失败的若干条经验法则：系统总是在多重故障的边缘运行，事故是多重因素叠加的结果，操作者实际上同时承担着维持生产与防范故障的双重角色，而事后归因往往把事故简化为某个…
其他2017 · 154 分 · 19 评入门 · 约 30 分钟编辑精选如何阅读论文
How to Read a Paper (2016)
介绍如何高效阅读学术论文，提出“三遍阅读法”：先快速浏览摘要、结论和图表把握概览，再理解主要内容，最后深入细节。
其他2017 · 715 分 · 80 评入门 · 约 10 分钟公元 1021 年欧洲人已在美洲存在的证据
Evidence for European presence in the Americas in AD 1021
这篇 Nature 论文用树木年轮定年，并结合公元 993 年宇宙射线事件标记，把纽芬兰 L'Anse aux Meadows 的 Norse 活动精确到公元 1021 年，为欧…
其他2021 · 655 分 · 391 评进阶 · 约 45 分钟一次通古斯级空爆摧毁了约旦河谷青铜时代城市 Tall el-Hammam
A Tunguska size burst destroyed Tall el-Hammam, Bronze Age city in Jordan Valley
研究约旦河谷青铜时代城市 Tall el-Hammam 被毁的原因。
其他2021 · 653 分 · 342 评入门 · 约 45 分钟