编程语言与软件工程2014macbeth.cs.ucdavis.edu
GitHub 中编程语言与代码质量的大规模研究
A Large Scale Study of Programming Languages and Code Quality in GitHub
进阶 · 约 50 分钟读完前置知识：统计回归基础、编程语言类型系统、Git/GitHub 数据
导读
这篇论文对 GitHub 上多种流行语言的大型仓库做统计分析，试图判断编程语言是否与代码缺陷率相关，并在控制项目年龄、开发者数、提交数和代码行数后给出回归结果。作者报告强类型略优于弱类型、静态类型略优于动态类型、函数式略优于过程式、托管内存略优于非托管，但效应较小，且方法受到评论者质疑。
HN 关注点：这类跨语言缺陷率比较是否可信，讨论焦点集中在缺陷计数方式、项目流行度混杂、语言分类和 GitHub 语言识别错误等方法问题。
HN 评论精选
这几条评论里：质疑 4
质疑质疑 bug 计数和回归推断，认为结果表不合常理。
作者查看了 20 种最流行语言加 TypeScript 各自在 GitHub 上 star 最多的 50 个仓库（排除 CSS、shell 和 vim），统计每个项目实际使用的语言；再通过 commit/PR 日志和关键词搜索统计各语言的 bug 数，未修复的 open issue 似乎不算。随后对项目年龄、开发者数、commit 数、代码行数做回归，得到语言与缺陷率相关表。这里有不少逻辑跳跃；结果表里 Perl 和 Ruby 一样可靠、且显著高于 Erlang 和 Java 等，也显得奇怪。
luu查看英文原文
The claim: “Most notably, it does appear that strong typing is modestly better than weak typing, and among functional languages, static typing is also somewhat better than dynamic typing. We also find that functional languages are somewhat better than procedural languages.” But how did they determine that? The authors looked at the 50 most starred repos on github, for each of the 20 most popular languages plus typescript (minus CSS, shell, and vim). For each of these projects, they looked at the languages used (e.g., projects that aren’t primarily javascript often have some javascript). They then looked at commit/PR logs to determine figure out how many bugs there were for each language used. As far as I can tell, open issues with no associated fix don’t count towards the bug count. Only commits that are detected by their keyword search technique were counted. After determining the number of bugs, the authors ran a regression, controlling for project age, number of developers, number of commits, and lines of code. That gives them a table (covered in RQ1) that correlates language to defect rate. There are a number of logical leaps here that I’m somewhat skeptical of. I might believe them if the result is plausible, but a number of the results in their table are odd. The table “shows” that Perl and Ruby are as reliable as each other and significantly more reliable than Erlang and Java (which are also equally reliable), which are significantly more reliable than Python, PHP, and C (which are similarly reliable), and that typescript is the safest language surveyed. They then aggregate all of that data to get to their conclusion. I find the data pretty interesting. There are lots of curious questions here, like why are there more defects in Erlang and Java than Perl and Ruby? The interpretation they seem to come to from their abstract and conclusion is that this intermediate data says something about the languages themselves and their properties. It strikes me as more likely that this data says something about community norms (or that it's just noise), but they don’t really dig into that. For example, if you applied this methodology to the hardware companies I’m familiar with , you’d find that Verilog is basically the worst language ever (perhaps true, but not for this reason). I remember hitting bug (and fix) #10k on a project. Was that because we have sloppy coders or a terrible language that caused a ton of bugs? No, we were just obsessive about finding bugs and documenting every fix. We had more verification people than designers (and unlike at a lot of software companies, test and verification folks are first class citizens), and the machines in our server farm spent the majority of their time generating and running tests (1000 machines at a 100 person company). You’ll find a lot of bugs if you run test software that’s more sophisticated than Quickcheck on 1000 machines for years on end. If I had to guess, I would bet that Erlang is “more defect prone” than Perl and Ruby not because the language is defect prone, but because the culture is prone to finding defects. That’s something that would be super interesting to try to tease out of the data, but I don't think that can be done just from github data.
质疑缺陷修复数受项目流行度和用户规模影响，不能直接归因于语言。
注意，流行语言的热门项目往往比冷门语言的热门项目有更多修复提交；可能只是因为它们用户更多，导致 bug 报告更多、社区压力更大。论文有趣，但很容易过度推断。
breuderink查看英文原文
> For those with positive coefficients we can expect that the language is associated with, ceteris paribus, a greater number of defect fixes. These languages include C, C++, JavaScript, Objective-C, Php, and Python. The languages Clojure, Haskell, Ruby, Scala, and TypeScript, all have negative coefficients implying that these languages are less likely than the average to result in defect fixing commits. Notice how top projects in popular languages do have tendency to have more fixes than top projects in more obscure languages. Perhaps these projects have simply more users, leading to more reported bugs and community pressure? Interesting paper, but it is all to easy to jump to conclusions.
质疑样本、项目年龄和 GitHub 语言识别错误削弱结论。
样本量太小，而且似乎没有充分加权项目年龄；老项目因规格变更多通常更容易出现缺陷。TypeScript 的结果完全错误：Bitcoin、litecoin、qBittorrent 都没有 TypeScript 代码，但 GitHub 却显示 Bitcoin 项目 74% 是 TypeScript。
ChrisCinelli查看英文原文
The sample size is too low. It also seem not to weight enough the age of the project. Older project are usually more prone to defect since the amount of changes of specs that they have to go through. The results for Typescript are completely wrong. Bitcoin, litecoin, qBittorrent do not have any Typescript code http://bitcoin.stackexchange.com/questions/22311/why-does-github-say-that-the-bitcoin-project-is-74-typescript Double Face palm!
质疑Python/Ruby 矛盾结果说明语言总体缺陷率比较不可靠。
Python 和 Ruby 的结果相反，但两者都是动态类型通用语言，非常相似，没有理由错误率天差地别。如果这项研究像徒劳之举，这就是冒烟枪。研究承认效应很小，唯一明确结论是缺陷类型与语言强相关：内存错误、并发错误等依赖语言原语，语言对特定缺陷类别比总体缺陷更重要。Clojure 是否天生比 C++ 缺陷率低？也许有趣，但 Python/Ruby 的结果说明只能啤酒桌上争论，不能靠数据定论。
numlocked查看英文原文
Python and Ruby come out with quite opposite results. There is absolutely no reason why those languages, on their own, should lead to wildly different error rates. Both dynamically typed, general purpose, very equivalent languages. So if this study seems intuitively like a fools-errand, that would be your smoking gun. The study admits the effects are small. The one clear conclusion is an obvious one: "Result 4: Defect types are strongly associated with languages; Some defect type like memory error, concurrency errors also depend on language primitives. Language matters more for specific categories than it does for defects overall." Does Clojure have an inherently lower defect rate than C++? It's arguably an interesting question to ask, but looking at the Python and Ruby results makes it clear, to my anyway, that we'll have to stick to arguing about it over beers, rather than finding the answer in data.
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2014Hacker News 讨论帖132 分 · 40 评
- 2015Hacker News 讨论帖22 分 · 11 评
同领域还值得读
Google 的软件工程（2017）
Software Engineering at Google (2017)
介绍了 Google 的软件工程实践，包括其单一代码库（monorepo）、几乎全部开发在主干进行而非分支、代码审查、构建系统 Blaze、协议缓冲区等工具与流程，并讨论了频繁重…
编程语言与软件工程2017 · 1,127 分 · 465 评 · 引用 15入门 · 约 45 分钟编辑精选Peter Naur：编程作为理论构建 (1985)
Peter Naur – Programming as Theory Building (1985)
Peter Naur 1985 年的经典论文《Programming as Theory Building》提出：编程的本质不是写代码或文档，而是在程序员头脑中构建对程序及其运行…
编程语言与软件工程2016 · 629 分 · 74 评入门 · 约 45 分钟编辑精选SciPy 1.0：Python 科学计算的基础算法
SciPy 1.0: fundamental algorithms for scientific computing in Python
这篇发表于 Nature Methods 的论文介绍了 SciPy 1.0——Python 科学计算生态的基石库，涵盖其核心算法模块（优化、积分、插值、线性代数等）、软件架构与社…
编程语言与软件工程2020 · 387 分 · 98 评入门 · 约 45 分钟编辑精选Lisp 的历史（1979）
History of Lisp (1979)
这是 Lisp 发明者 John McCarthy 于 1979 年为 HOPL（编程语言史）会议撰写的回顾文章，讲述 Lisp 的起源、设计动机与早期演进，文中也涉及 if 表…
编程语言与软件工程2018 · 352 分 · 44 评入门 · 约 45 分钟编辑精选使用 NumPy 进行数组编程
Array Programming with NumPy
这篇 Nature 文章介绍 NumPy 的数组编程模型及其在科学计算中的核心作用：以 ndarray 为中心，通过向量化操作让 Python 高效处理多维数组。
编程语言与软件工程2020 · 318 分 · 134 评入门 · 约 40 分钟编辑精选