经济与社会科学2024papers.ssrn.com
生成式 AI 对高技能工作的影响：面向软件开发者的实验
Effects of Gen AI on High Skilled Work: Experiments with Software Developers
进阶 · 约 35 分钟读完前置知识：随机对照实验、软件开发流程、生成式 AI 工具
导读
这篇论文通过随机对照实验研究生成式 AI（如 Copilot）对高技能软件开发者生产力的影响，发现使用 AI 工具的开发者完成任务的速度提升约 26%。研究关注 AI 对不同经验水平开发者的差异化影响。
HN 关注点：HN 读者长期争论 AI 编程工具的真实生产力收益，这篇实验研究提供了量化数据，讨论焦点集中在经验水平差异、衡量方式是否合理以及技术债等隐性成本。
HN 评论精选
这几条评论里：质疑 2 补充 2
质疑AI 提效的代价是资深开发者承担技术债，且最热衷 AI 的往往是不关心技术的开发者
我怀疑研究是否考虑了技术债：经验较少的开发者用 AI 产出代码后，资深开发者要花大量时间收拾烂摊子——我就在研究涉及的公司之一亲身经历过这种情况。另外，我发现对 AI 最热衷的往往是那些对技术本身兴趣不大、只关心交付的开发者，而 PM 们很欣赏这类人。
alternatex查看英文原文
I wonder if the study includes the technical debt that more experienced developers had to tackle after the less experienced devs have contributed their AI-driven efforts. Because my personal experience has involved a lot of that in one of the companies listed in the study. Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though.
补充开发者被迫身兼数职、环境过于复杂，才不得不依赖 AI
我有时怀疑 IT 从业者质量下降是因为公司不断把更多角色压到每个员工身上以裁减人手。开发、运维、安全原本是专职岗位，后来有了 DevOps，一些企业理解为只需 2/3 的人手；再后来 DevSecOps，又变成 1/3。这不是否定 shift-left 模式本身，而是高管为削减成本拿奖金的必然结果。现在新开发者要面对极度复杂的多微服务环境、学习现有代码库、掌握 5 个工具的 CI/CD 流水线、还要兼任 DBA，同时保持稳定发布节奏。他们用 ChatGPT 应付有什么好惊讶的？只要 IT 公司还在靠裁员让曲线变好看，这种情况就会持续。
ang_cire查看英文原文
I sometimes wonder about whether the decline in IT worker quality is down to companies trying to force more and more roles onto each worker to reduce headcount. Developers, Operations, and Security used to be dedicated roles. Then we made DevOps and some businesses took that to mean they only needed 2/3 of the headcount, rather than integrating those teams. Then we made DevSecOps, and some businesses took that to mean they only needed 1/3 the original roles, and that devs could just also be their operations and appsec team. That's not a knock on shift-left and integrated operations models; those are often good ideas. It's just the logical outcome of those models when execs think they can get a bigger bonus by cutting costs by cutting headcounts. Now you have new devs coming into insanely complex n-microservice environments, being asked to learn the existing codebase, being asked to learn their 5-tool CI/CD pipelines (and that ain't being taught in school), being asked to learn to be DBAs, and also to keep up a steady code release cycle. Is anyone really surprised they are using ChatGPT to keep up? This is going to keep happening until IT companies stop cutting headcounts to make line go up (instead of good business strategy).
质疑研究只衡量了 Copilot 补全，资深开发者若善用 AI 于编码前的思考环节，收益可能更大
关键在于你衡量什么。这些研究只看了 Copilot 的使用。作为资深工程师，Copilot 对我来说比没用还糟：我大部分时间花在理解问题空间、环境约束和思考要写的代码上，等我开始敲代码时已经知道要写什么，自动补全只会干扰我。但 AI 在编码前的那些思考环节极其有用，有时精心设计的 prompt 就能拿到初稿，之后与 LLM 结对快速解答各种小问题也非常有帮助。所以与报告相反，我认为善用 AI 的资深开发者反而可能获益更多。
dimal查看英文原文
It matters what you measure. The studies only looked at Copilot usage. I’m an experienced engineer. Copilot is worse than useless for me. I spend most of my time understanding the problem space, understanding the constraints and affordances of the environment I’m in and thinking about the code I’m going to write app. When I start typing code, I know what I’m going to write, and so a “helpful” Copilot autocomplete is just distraction for me. It makes my workflow much much worse. On the other hand, AI is incredibly useful for all of those steps I do before actually coding. And sometimes getting the first draft of something is as simple as a well crafted prompt (informed by all the thinking I’ve done prior to starting. After that, pairing with an LLM to get quick answers for all the little unexpected things that come up is extremely helpful. So, contrary to this report, I think that if experienced developers use AI well, they could benefit MORE than inexperienced developers.
补充AI 的帮助程度取决于所用技术是否熟悉，学习新技术时收益最大
26% 的生产力提升与我的经验相符。我认为他们应该探索的一个维度是：你是在用新技术还是已熟悉的技术。AI 对我学习新语言/框架时帮助大得多。
danielvaughn查看英文原文
A 26% productivity increase sounds inline with in my experience. I think one dimension they should explore is whether you're working with a new technology or one that you're already familiar with. AI helps me much more with languages/frameworks that I'm trying to learn.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2024Hacker News 讨论帖335 分 · 511 评
同领域还值得读
一切的回报率，1870–2015
The Rate of Return on Everything, 1870–2015
利用1870–2015年多国的长期数据，估算股票、住房等各类资本的回报率，并将其与经济增长率进行比较。
经济与社会科学2019 · 487 分 · 260 评进阶 · 约 100 分钟编辑精选一切资产的回报率，1870–2015
The Rate of Return on Everything, 1870–2015
构建了 1870–2015 年 16 个国家的长周期资产回报数据集，覆盖股票、住房、政府债券和短期票据，系统比较各类资产的真实回报与波动性。
经济与社会科学2018 · 215 分 · 151 评进阶 · 约 120 分钟编辑精选一切资产的回报率，1870–2015（2019）
The Rate of Return on Everything, 1870–2015 (2019)
这篇发表于《经济学季刊》的论文构建了 1870–2015 年间 16 个主要发达国家的股票、债券、房地产等资产收益率的长周期数据库。
经济与社会科学2024 · 190 分 · 148 评进阶 · 约 80 分钟编辑精选想法是否越来越难找？
Are ideas getting harder to find?
探讨“想法是否越来越难找到”：通过衡量多个领域的“研究生产率”，发现维持相同的增长需要投入指数级增长的研究资源，即研究效率在持续下降。
经济与社会科学2016 · 182 分 · 106 评入门 · 约 60 分钟编辑精选比特币与区块链的经济极限
The Economic Limits of Bitcoin and the Blockchain
芝加哥大学 Eric Budish 的论文从经济学角度分析比特币与区块链的根本限制：核心论点是只要攻击网络（双花或砸盘做空）的收益超过挖矿成本，51% 算力攻击就有利可图，而挖矿…
经济与社会科学2018 · 176 分 · 37 评进阶 · 约 90 分钟编辑精选