经济与社会科学2022papers.ssrn.com
可预测的糟糕投资：来自风险投资家的证据
Predictably Bad Investments: Evidence from Venture Capitalists
进阶 · 约 55 分钟读完前置知识：机器学习基础（XGBoost/时间切分）、风险投资与创业融资基础、文本特征与TF-IDF
导读
论文用 PitchBook 数据训练 XGBoost 模型，以 2014 年前的创业投资为训练集、2014–2016 年的投资为时间隔离的测试集，预测哪些创业公司会成功，并据此声称风险投资人的决策存在系统性偏差（可预测的糟糕投资）。模型最重要的特征是公司及 CEO 的文字描述，被表示为 unigram/ngram 的 TF-IDF 计数。
HN 关注点：HN 读者既被“ML 能否战胜 VC”吸引，也集中质疑其方法论：特征可能包含决策时点之后才有的信息，关键变量靠假设填补，模型可能只是学到了事后才知道的行业冷热。
HN 评论精选
这几条评论里：质疑 4
质疑时间切分做得对，但特征可能包含决策后信息（前视偏差），且未解释模型学到什么。
论文训练 XGBoost 预测 2014–2016 年创业投资的结果，训练集是 2014 年前的投资，特征来自 PitchBook。文中说最重要的特征是对公司和 CEO 的文字描述，用 unigram 和 ngram 的 TF-IDF 计数表示。论文至少避免了最明显的方法论问题：用按时间切分的留出期（2014–2016）而不是随机留出样本，否则模型会学到未来趋势——即便你挑不出哪些公司会成功，知道 2026 年哪些公司成功也会极大帮助今天的投资决策。但论文似乎没有限制只使用投资决策当时已记录的信息：CEO 的描述会随时间变化，公司要上市时可能会强调 CEO 有 MBA，而种子轮就死掉的公司这些细节可能根本不会出现在文本里；连创始人名单都可能被事后修改（特斯拉是著名例子）。论文也没有告诉我们模型究竟学到了什么，因此必须考虑其他竞争性解释。
syllogism查看英文原文
The paper trains an XGBoost model to predict investment outcomes for startup investments from the time period 2014-2016. The model is trained on investments prior to 2014, and uses features from Pitchbook. It says the most important features are text descriptions of the company and CEO, which are featurized as unigram and ngram TF-IDF counts. The paper avoids the very first methodological problem one would check for, which is using a held-out set that is mixed in time. That is: at least it uses a heldout _period_ (2014-2016), instead of individually held-out samples. Using individually held-out samples would let the model learn about future trends. If you know some companies that will be successful in 2026, that will help a lot in deciding what to invest in today, even if you can't pick those companies. So at least the paper doesn't do that. However, the paper doesn't seem to have limited itself to information written and documented about the companies at the time of the investment decisions. Descriptions about the CEO will change over time. Once a company is looking to go public, maybe they emphasise that their CEO has an MBA. If your company died at seed, maybe this detail isn't included in the text. Even the people listed as the founders of the company is something that might be revised over time (Tesla is a famous example). The paper really doesn't give us much insight into what the model learns. This means that we need to consider competing explanations for the model's accuracy --- we're not forced to accept the author's conclusion that the investments were indeed "predictably bad" in a way that could be profitably implemented as a trading strategy. By itself, the fact that a black-box machine learning model with a large number of features can distinguish between two classes on a held-out set really isn't very strong evidence that the model has learned something of practical importance, that will generalise beyond the train and held-out sets you've collected. You need to show us what it's actually learned. If it's supposed to identify wolves, is it looking at the canines or the snow? For a study like this, I'd want to see feature pruning down to where the model is still accurate but with only a few textual features, and then show us what those features are. If you can't do that in a way that actually leads to human insight, it's extremely unlikely that the model is learning anything real.
质疑核心因变量靠大量假设插补（缺失即估值为零、固定稀释因子），结论可疑，属数据挖掘。
作者原文写道：“其中若干数值在数据中并未提供，因此需要插补/假设。首先，t=2 时点的公司估值定义为截至 2020 年底最近一轮‘后期’融资的估值；若不存在此类轮次，估值设为零；若存在但 PitchBook 未提供估值，我也设为零。其次，PitchBook 通常给出早期投资人的初始持股，若没有则我假设一个早期持股比例。最后，我假设统一的 0.75 稀释因子。”我对影响预测目标最大的变量做了一堆假设，然后发现算法说大家都错了。这听起来就像刚学 ML 就跑去写股市预测器、然后才明白投资界所谓“数据挖掘”是什么意思的人。
drdrek查看英文原文
"Several of these quantities are not provided in the data and therefore need to be imputed assumed. First, the valuation of the firm at t=2 in model time is defined as the latest associated with a “late-stage” round by the end of 2020. If no such rounds exist the company valuation is set to zero. If such a round exists but the valuation is provided by Pitchbook, I set it to zero. Next, Pitchbook typically provides the initial stake of the early stage investors. If it does not then I assume the early stake is Finally, I assume a constant dilution factor of .75." I've made assumptions about the thing that effects what I try to predict the most and have found that my algorithm says people are wrong. Sounds like those people that first learn about ML than immediately runs off to build a stock market predictor before inevitably learning about the investment worlds definition to "Data mining"
质疑模型可能只是在复现“融资规模”这一信号，应做剔除该特征的稳健性检验。
我的朴素看法是，VC 会向他们认为会成功的公司投更多钱（即更大的融资轮）。模型有没有可能部分抓的正是这个信号？也就是说结果其实只说明“VC 投得越多的公司越成功”，而这很可能是因为 VC 本身有合理的判断标准。论文里似乎没讨论这点，不知道如果把融资轮规模从输入特征中剔除，结果是否还这么强。
alexedw查看英文原文
My naive belief is that VC's invest more in companies that they think will succeed (i.e. larger funding rounds). Is it possible the model is partially latching onto this, meaning their result could actually be just saying "companies that VC's back more are also the one's that succeed", presumably because VC's have some of their own reasonable criteria for doing this. I didn't see this discussed in the paper, I wonder if it's results would be as strong if they excluded size of funding rounds as an input.
质疑模型“跑赢 VC”是因为训练集已含同一时代哪些赛道成功的事后信息，而真实投资人只能预判。
这论文很有意思，但我不太相信作者所称“他们的 ML 模型能比 VC 做出更好的投资决策”。模型当然能轻松跑赢真实投资人的回报：它的训练集包含了科技行业未来的硬数据，而真实投资人没有未来的数据，只能猜。数据集包含每家公司的描述，模型可以从训练集学到 VR、3D 打印、企业协作、清洁技术等领域在 2014 年是好的还是坏的投资方向，然后把这个知识应用到同一时代的测试集上（补充：正如另一位评论者引用的，“论文说最重要的特征是对公司和 CEO 的文字描述，用 unigram 和 ngram 的 TF-IDF 表示”）。做真实决策的投资人无法预知 VR、3D 打印、清洁技术、低代码/无代码这些领域会火还是会凉，只能猜。同样重要的是，模型并不知道今天被吹捧为热门的领域中哪些事后会被证明真的热——它只知道 2014 年哪些热。
yodon查看英文原文
This is a super interesting paper, but I wouldn't put much trust in the author's suggestion that their machine learning model knows how to make better investment decisions than VC's do. Of course their model soundly beat the returns of real investors. Its training dataset contained hard data about the future of the tech industry. Real investors don't have hard data on the future, they have to guess. The dataset includes descriptions of each of the firms. The model gets to learn from the training dataset whether sectors like VR, 3D printing, business collaboration, clean tech, or etc. were good or bad areas to invest in in 2014, and then gets to apply that knowledge to the test dataset which covers the same epoch. (EDIT ADDED: to quote another commenter here, "[The paper] says the most important features are text descriptions of the company and CEO, which are featurized as unigram and ngram TF-IDF counts.") Real investors making real investment decisions don't know in advance whether sectors like VR or 3D printing or clean tech or low-code/no-code or etc. are going to be hot or duds, they have to guess. Equally importantly, this model doesn't know which of the sectors being touted as hot investment opportunities today will, in hindsight, prove to have been hot today. It only knows which ones were hot in 2014.
历次讨论
- 2022Hacker News 讨论帖118 分 · 60 评
- 2024Hacker News 讨论帖43 分 · 6 评
同领域还值得读
一切的回报率，1870–2015
The Rate of Return on Everything, 1870–2015
利用1870–2015年多国的长期数据，估算股票、住房等各类资本的回报率，并将其与经济增长率进行比较。
经济与社会科学2019 · 487 分 · 260 评进阶 · 约 100 分钟编辑精选一切资产的回报率，1870–2015
The Rate of Return on Everything, 1870–2015
构建了 1870–2015 年 16 个国家的长周期资产回报数据集，覆盖股票、住房、政府债券和短期票据，系统比较各类资产的真实回报与波动性。
经济与社会科学2018 · 215 分 · 151 评进阶 · 约 120 分钟编辑精选一切资产的回报率，1870–2015（2019）
The Rate of Return on Everything, 1870–2015 (2019)
这篇发表于《经济学季刊》的论文构建了 1870–2015 年间 16 个主要发达国家的股票、债券、房地产等资产收益率的长周期数据库。
经济与社会科学2024 · 190 分 · 148 评进阶 · 约 80 分钟编辑精选想法是否越来越难找？
Are ideas getting harder to find?
探讨“想法是否越来越难找到”：通过衡量多个领域的“研究生产率”，发现维持相同的增长需要投入指数级增长的研究资源，即研究效率在持续下降。
经济与社会科学2016 · 182 分 · 106 评入门 · 约 60 分钟编辑精选比特币与区块链的经济极限
The Economic Limits of Bitcoin and the Blockchain
芝加哥大学 Eric Budish 的论文从经济学角度分析比特币与区块链的根本限制：核心论点是只要攻击网络（双花或砸盘做空）的收益超过挖矿成本，51% 算力攻击就有利可图，而挖矿…
经济与社会科学2018 · 176 分 · 37 评进阶 · 约 90 分钟编辑精选