关于测试驱动开发效果的外部重复实验

导读

这篇论文是一项关于测试驱动开发（TDD）效果的外部复现研究，参与者在 Bowling Kata、Mars Rover 类 API 等编程任务上开发，并考察 TDD 的影响。评论显示该研究样本仅 20 名以学生为主的新手、采用便利抽样，因此更像是在简单任务和初学者场景下检验 TDD，难以据此推断真实项目中的开发时间或代码质量。

HN 关注点：TDD 是否真的有效，讨论焦点集中在研究的外部效度：小样本、学生参与者、简单题目是否足以支撑对 TDD 效果的结论，以及 TDD 是否更适合特定人和特定任务。

关键词　TDD外部复现软件工程研究样本偏差

HN 评论精选

这几条评论里：质疑 1　补充 1　亲历 2

质疑批评研究样本小、便利抽样、参与者多为新手且任务过于简单，认为无法推广到真实开发。

和大多数我见过的软件开发研究一样，这项研究有严重缺陷，配不上 HN 上这个耸动标题。样本量很小（20 名学生）；参与者是便利抽样（研究者班上的学生）；大多数人没有职业经验（6 人有职业经验，只有 3 人超过两年）；编程题很琐碎（Bowling Kata 和一个“复杂度相当”的 Mars Rover API 问题）。也许你能用它推断 TDD 如何影响新手做简单算法题；但考虑到样本量小且是便利抽样，我甚至不确定能否得出这么多结论。它无法说明 TDD 是否影响真实开发中的开发时间或代码质量。

jdlshore查看英文原文

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two years' experience.) * The programming problems were trivial. (The Bowling Kata and an 'equivalent complexity' Mars Rover API problem.) Maybe, maybe you could use this to draw conclusions about how TDD affects novices working on simple algorithmic problems. Given the tiny sample size and sampling by convenience, I'm not sure you can even draw that much of a conclusion. But it won't tell you anything about whether or not TDD impacts development time or code quality in real-world development.

补充认为 TDD 很大程度取决于个人工作风格，真正重要的是有没有测试，反对把 TDD 当成唯一正确路径。

TDD 与“先写代码后写测试”会不会很大程度上是个人偏好？我个人觉得，在写出可工作的代码之后更容易写好测试。事先我只有一两个模糊想法，直到写出来、反复调试之后，才能列出具体的真实测试场景。但我知道有些人的思维方式不同：他们会先大量思考场景，再把所有情况想清楚后写代码。对他们来说，TDD 作为一种理念似乎更合适。我认为真正的鸿沟在于“没有测试的代码”和“有测试的代码”之间。我一直不理解 TDD 作为 XP 实践背后那种近乎宗教式的狂热。就像结对编程：如果它适合你和你的编码风格，那很好；但别强加给我，也别表现得像它是通向整洁代码的唯一真理。

geerlingguy查看英文原文

Could it be that TDD vs. tests-after-code is a highly personal thing? I personally find it easier to write good tests after I've coded something functional. Before hand, I know one or two fuzzy ideas of what I want to accomplish, but I can't list out the concrete, real-world test scenarios until after I've coded something, poked and prodded it, etc. But I know some people are wired differently; they'll think a lot more about scenarios first, then code after they have everything accounted for. For them, TDD as a philosophy seems more fitting. I think the chasm exists between _untested_ code and code that has tests. I've never understood the seemingly-religious zealotry behind TDD as an XP practice. Just like pair programming... if it works for you and your coding style, awesome. But don't force it down my throat or act like it's the One True Path to clean code.

亲历亲历：TDD 适合时间/金钱等易回归逻辑，不适合快速变化的 UI；UI 更适合稳定后写确定性测试。

我的经验是，TDD 在处理时间、金钱这类出了名容易“打地鼠”的函数时值得做：抓回归所省下的时间远超写测试的成本。相反，对基于 UI 的工作，TDD 对我一直是浪费时间，因为要正确暴露待测功能成本太高，而且需求和设计变化太快，不划算。后一种情况下，等需求和实现稳定后，针对 mock 数据写一些确定性 UI 测试，对防止回归有效得多。

tspike查看英文原文

My experience has been that TDD is worthwhile when working with notoriously slippery whack-a-mole functions like handling time or money. The time saved by catching regressions vastly outweighs the time taken to implement the tests. In contrast, TDD has been a waste of time for me for UI-based work, as the effort needed to properly expose the functionality under test is too great and the requirements and design change too quickly to be worth it. In the latter case, writing some deterministic UI tests against mock data after the requirements and implementation have settled has proven much more effective in preventing regressions.

亲历从企业实践出发，认为在 QA 被削弱、持续集成和逃逸缺陷压力下，必须大量投入自动化测试，TDD 熟练后可降低成本。

过去 18 个月，一股股管理风潮扫过我们这家 Scrum 企业：裁掉 QA 团队、开发团队成为二级生产支持、进入持续集成天堂。那些不持续测试的团队，每次发布都有大量逃逸缺陷；这些开发者一直处于救火状态，很痛苦。我看到这导致他们的排期被压缩，并出现更鲁莽的行为，比如要求把发布推到假期期间，而 bug 可能带来严重财务后果。在我看来，在我这样的环境里，开发者不能再躲在 QA 团队这类官僚机制后面掩盖无能，不花大量开发时间写自动化测试简直是疯了。你应该把 70% 的开发时间用于写测试和做 DevOps，30% 写功能。我在评论里看到很多抱怨写测试太花时间。第一，TDD 是一项技能，熟练后不会像你想的那么耗时；第二，当逃逸缺陷成为组织里的耻辱标记时，我认为你别无选择，只能全面测试。

EdSharkey查看英文原文

The * fire your QA team * dev team is the level 2 production support, and * get to continuous integration nirvana management fads have been sweeping through my Scrum enterprise for the last 18 months. Teams that aren't testing constantly, well, they've got tons of escape defects on every release. And those devs are constantly in fire-fighting mode, it's miserable for them. And I see that leading to compressed schedules for them and more reckless behavior like asking to push their releases during the holidays where there could be severe financial consequences to bugs. As far as I'm concerned, in an environment like mine, where developers can no longer hide their incompetence behind bureaucracy like a QA team, it is official insanity to not spend inordinate amounts of development time writing automated tests. You should be spending 70% of your dev time writing tests and doing devops and 30% writing features. I read in these comments a lot of bellyaching about how much time it takes to write tests. First, TDD is a skill that you can get good at, and it won't take as much time as you think once you get good. Second, I just don't think you have a choice to not test comprehensively when escape defects become a mark of shame in the organization.

历次讨论

2016Hacker News 讨论帖406 分 · 323 评

HN 二十年，最值得读的论文。

朋友分享的书单

专题阅读路线

本周最多人读

我的书单

必读 30 篇

论文库

这些论文是怎么挑出来的

全量扫一遍

只留论文本身

写导读、译评论

关于测试驱动开发效果的外部重复实验

导读

HN 评论精选

相关资源

历次讨论

同领域还值得读

Google 的软件工程（2017）

Peter Naur：编程作为理论构建 (1985)

SciPy 1.0：Python 科学计算的基础算法

Lisp 的历史（1979）

使用 NumPy 进行数组编程

打赏支持

关注向阳乔木