编程语言与软件工程2019arxiv.org
编程语言对代码质量的影响
On the Impact of Programming Languages on Code Quality
进阶 · 约 60 分钟读完前置知识：统计学基础、回归分析、软件度量基础
导读
这篇论文是对 2014 年一项“编程语言影响代码质量”研究的复现研究。作者对原始数据重新清洗并用改进的统计模型分析后发现，原论文 11 种语言中有 7 种的显著性不再成立，即使显著，实际影响也极小——语言选择对缺陷率的总体影响不到 1%。
HN 关注点：HN 读者长期争论“哪种语言 bug 更少”，这篇复现研究几乎推翻了常被引用的原始结论，讨论焦点集中在统计方法、混杂因素以及“代码质量”该如何衡量。
后来怎么样
作为对 2014 年编程语言与代码质量研究的复现研究，其结论（多数语言效应不显著、总体影响不足 1%）成为该领域常被引用的证据，说明语言选择对缺陷率影响甚微。
HN 评论精选
这几条评论里：支持 1 质疑 2 补充 1
补充澄清这是复现研究且基本复现失败，别混淆原论文与本文的结论
讨论中很多人把原论文的结论当成了这篇论文的结论。这是一项复现研究，开头陈述的是原论文发现了什么，而那并不是本文的发现。总体而言这是一次复现失败，原因包括多种分类错误。论文原话：'再分析未能验证 [18] 的大部分结论。经过多步数据清洗和改进的统计建模，11 种语言中有 7 种的显著性被推翻；即使关联在统计上显著，其实际意义也很小。'
scott_s查看英文原文
A common thread I'm seeing in the discussion here is confusing the claims from the original paper with conclusions made by this paper. This is a replication study, and it starts by stating what the original paper found. What the original paper found is not what this paper finds. For the most part, this is a replication failure, for a myriad of reasons, including several kinds of misclassification. From this paper: The reanalysis failed to validate most of the claims of [18]. As Table 6(d-f) shows, the multiple steps of data cleaning and improved statistical modeling have invalidated the significance of 7 out of 11 languages. Even when the associations are statistically significant, their practical significance is small. The reference [18] is https://dl.acm.org/citation.cfm?doid=2635868.2635922, which is the original paper that this paper is a replication study of.
质疑研究未控制经验、社区文化、选型动机等混杂因素，关联不等于因果
他们只做了关联分析，没有控制很多因素。比如 Python/JS/PHP/Java/C++ 都是很常见的入门语言，平均经验水平可能被拉低，而 Golang 使用与年龄强相关，年龄与 bug 数负相关，构成混杂因素。Ruby 社区对单元测试的执念也可能影响 bug 水平。C/C++ 因内存性能被选用，这些权衡是双刃剑，自然与更多 bug 相关——以'缺陷数'为主要指标会忽略语言被选择的原因。愿意主动学习小众语言的工程师可能本身水平更高。
robbrit查看英文原文
Seems like they're just doing associations, and not controlling for a lot of factors. For example, you can come up with some narratives to explain a lot of these findings: * Python, JS, PHP, Java, C++ are all very common first languages; I'd imagine the average experience level for these is dragged down a bit compared to others like Golang, Haskell, or Scala. One of the recent Stack Overflow Developer surveys found Golang use to be strongly correlated with age, and since I'm assuming bug count would negatively correlate with age you'd see a confounding factor here. * Ruby's community has an obsessive culture towards unit testing, that could impact bug levels. * The trade-offs that make C and C++ attractive such as memory performance are double-edged swords and will naturally correlate with more bugs. For example, I work a fair bit in embedded systems which use MISRA C out of necessity; I've also worked in large scale data processing which chooses C++ for its memory mapping performance over Java or Go. If "number of defects" is your primary measure then you're ignoring some of the reasons languages are chosen. Sometimes a language that minimizes bugs is not a viable option. * I'd guess there is a skill correlation here: an engineer who is willing to seek out a specific less-used language and learn to use it effectively is probably more likely (not guaranteed) to do a better job than one who is just going with the default choice. It'd be very interesting to see a regression done that accounts for these factors.
质疑每行 bug 数无法衡量代码质量，总行数和缺陷性质同样重要
问题在于'代码质量'不只是每行代码的 bug 数，可维护性、可扩展性、可读性、表达力都算在内。假如用语言 B 要写 10 万行而语言 A 只要 1 万行，即使 A 每行缺陷率高两倍，总缺陷仍少五倍。另外缺陷的性质也不同：允许远程执行漏洞的缺陷、被忽略并传播的错误结果、带诊断信息地终止，三者不可等同。
kazinator查看英文原文
The problem is that "code quality" isn't just bugs per line of code. Maintainability, extensibility, readability, expressivity all factor into it. Suppose I have to write 100,000 lines in language B to write something doable in 10,000 lines of A. Who cares if the defect rate is twice as high per line of A: I still have five times fewer defects due to ten times fewer LOC's. Another thing to consider is the nature of defects. A defect that allows a remote execution exploit is not the same as an incorrect result being ignored and propagating to later calculations, is not the same as terminating with a diagnostic.
支持语言对 bug 率的总体影响不到 1%，语言间的相对差异意义有限
原论文最具实用价值的结论依然成立：语言选择对 bug 率的影响不到 1%。原论文明确说项目活跃度解释了绝大部分偏差，语言是第二接近的预测因子但不到 1%。所以即便是 C++ 与 Haskell 之间约 25% 的相对差异，也落在这 1% 之内——就像一项研究说 Nike 比 Adidas 让你跑快 25%，而跑鞋对你速度的影响总共不到 1%。
pron查看英文原文
The most practically relevant finding of the original paper remains in effect: the choice of language has an effect of less than 1% on the bug rate[1]. The original paper explains: "With effects coding, each coefficient indicates the relative effect of the use of a particular language on the response as compared to the weighted mean of the dependent variable across all projects." So the entire discussion is on the relative effect of particular languages relative to the total effect of language choice, which is less than 1%. The original paper reported stronger relative effects for particular languages, while this one found the effects to be smaller, and in many cases they disappeared altogether, thus summarizing the effects of the languages as "exceedingly small." So even the comparison of C++ vs. Haskell, that has ~25% difference, is within that 1%. This is similar to a study comparing the effect of running shoes on running speed that shows that Nike positively affects your running speed 25% more than Adidas, to the extent running shoes affect your speed at all, which is less than 1%. ----- [1] One should take care not to overestimate the impact of language on defects. While these relationships are statistically significant, the effects are quite small. In the analysis of deviance... we see that activity in a project accounts for the majority of explained deviance... The next closest predictor, which accounts for less than one percent of the total deviance, is language. (https://web.cs.ucdavis.edu/~filkov/papers/lang_github.pdf)
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2019Hacker News 讨论帖264 分 · 133 评
同领域还值得读
Google 的软件工程（2017）
Software Engineering at Google (2017)
介绍了 Google 的软件工程实践，包括其单一代码库（monorepo）、几乎全部开发在主干进行而非分支、代码审查、构建系统 Blaze、协议缓冲区等工具与流程，并讨论了频繁重…
编程语言与软件工程2017 · 1,127 分 · 465 评 · 引用 15入门 · 约 45 分钟编辑精选Peter Naur：编程作为理论构建 (1985)
Peter Naur – Programming as Theory Building (1985)
Peter Naur 1985 年的经典论文《Programming as Theory Building》提出：编程的本质不是写代码或文档，而是在程序员头脑中构建对程序及其运行…
编程语言与软件工程2016 · 629 分 · 74 评入门 · 约 45 分钟编辑精选SciPy 1.0：Python 科学计算的基础算法
SciPy 1.0: fundamental algorithms for scientific computing in Python
这篇发表于 Nature Methods 的论文介绍了 SciPy 1.0——Python 科学计算生态的基石库，涵盖其核心算法模块（优化、积分、插值、线性代数等）、软件架构与社…
编程语言与软件工程2020 · 387 分 · 98 评入门 · 约 45 分钟编辑精选Lisp 的历史（1979）
History of Lisp (1979)
这是 Lisp 发明者 John McCarthy 于 1979 年为 HOPL（编程语言史）会议撰写的回顾文章，讲述 Lisp 的起源、设计动机与早期演进，文中也涉及 if 表…
编程语言与软件工程2018 · 352 分 · 44 评入门 · 约 45 分钟编辑精选使用 NumPy 进行数组编程
Array Programming with NumPy
这篇 Nature 文章介绍 NumPy 的数组编程模型及其在科学计算中的核心作用：以 ndarray 为中心，通过向量化操作让 Python 高效处理多维数组。
编程语言与软件工程2020 · 318 分 · 134 评入门 · 约 40 分钟编辑精选