系统与数据库2014domino.research.ibm.com
虚拟机与 Linux 容器性能比较的最新研究
An Updated Performance Comparison of Virtual Machines and Linux Containers
进阶 · 约 60 分钟读完前置知识：Linux 容器与 cgroups 基础、KVM 虚拟化原理、网络 NAT 与文件系统基础
导读
这篇 IBM 论文对虚拟机（KVM）与 Linux 容器（Docker）进行了更新的性能对比测试。结果显示容器本身开销很小，但 Docker 默认配置中的 NAT 网络和 AUFS 存储层会带来显著性能损失，虚拟机在部分测试中也接近原生性能。
HN 关注点：容器与虚拟机的真实性能差异，讨论焦点集中在 Docker 的 NAT/AUFS 开销来源以及测试方法的合理性。
HN 评论精选
这几条评论里：质疑 1 补充 2 亲历 1
补充指出标题误导，论文真正揭示的是 Docker NAT 和 AUFS 的显著 I/O 开销
标题有点误导。拿 Linux 容器和原生性能比其实没什么意思，cgroups、chroot 这些机制不太可能有明显开销。实际上论文比较的是不同 Docker 配置与原生（以及虚拟机）的性能，能得出的洞见是本地 NAT 和 AUFS 引入的 I/O 开销——结果证明这两者的开销相当可观。另外图 17 显示容器在 NAT/AUFS 之前还有约 2% 的固有开销，其来源值得探究。
usaar333查看英文原文
A bit of a misleading title. Comparing linux containers to native performance wouldn't be terribly interesting; there is little reason to believe there is any nontrivial overhead from using cgroups, chroot, etc. Instead, this is comparing different Docker configurations to native performance (to Virtual Machines as well). The insights one can get is the amount of I/O overhead that is introduced by using local NAT as well as AUFS. (turns out NAT and AUFS overhead are quite significant.) Finally, figure 17 is a bit interesting. It implies that there is some inherent overhead (before NAT/AUFS) of 2% from the container; it'd be interesting to find out what the source of that is. /aside: putting percentage loss and absolute numbers on the same graph (figure 16) is incredibly confusing.
质疑质疑 KVM 测试配置不当，认为 hypervisor 之间的对比更有价值
在 Linpack 测试中，KVM 虚拟机应该配置为继承宿主机的 CPU 扩展指令集（或挑选与该测试相关的扩展）。除此之外，更新一版 KVM vs Xen vs VMware 的对比会更有意思。如果不需要运行其他操作系统，进程隔离（jails/zones/容器等）总是能带来更好的性能。
sergiolp查看英文原文
On the linpack test, the KVM domain should have been configured for inheriting the CPU extensions from the host (or, alternatively, cherry picking the extensions relevant for such test). For the rest, I think an updated KVM vs. Xen vs. VMWare comparison would have been way more interesting. If you don't need to run other OSes, process tagging (jails/zones/containers/whateveritscalledonyouros) will always give you better performance.
补充惊讶于 vETH+NAT 造成约 20% 的网络性能损失，推测是内核拷贝所致
有意思的是使用 vETH 和 NAT 会带来约 20% 的额外延迟和约 20% 的吞吐下降。这符合预期吗？我本以为它们不会有明显影响，毕竟硬件网卡应该是瓶颈？也许内核为了实现这些功能在做内存拷贝？不过 10Gbps 网卡相当快，在更常见的 100M/1G 网卡上这些损失可能不那么显著。
tinco查看英文原文
Interesting that using vETH and NAT causes around 20% extra latency and around 20% less throughput. Is this expected from these technologies? I was expecting them to have no significant impact at all, since surely the hardware NIC would be the bottleneck? perhaps the kernel is doing some memcpy'ing to achieve these features? Then again, 10gbps NIC is quite fast, perhaps this performance losses are less significant with more reasonable 100mbit/1gbit interfaces?
亲历亲历者观点：虚拟化有代价，应按场景权衡而非默认“免费”
虚拟化是有代价的，任何设计都应在代价与收益之间权衡。我设计过低延迟、高 QPS 的系统，这种场景裸机无可替代。但如果你要管理 5000 台机器，就得权衡虚拟环境的便利与多买 600 台服务器的成本。多数情况下，性能差异相比安全、维护和管理人力来说只是噪音。遗憾的是很多人以为虚拟化或容器是“免费”的，做出缺乏依据的决策，在系统规模化后产生真实影响。没有免费的午餐。
kator查看英文原文
Virtualization costs something and the goals of any design should be to balance that cost against the benefits. I've designed systems that have crazy low-latency and high QPS needs and nothing beats bare metal for these systems. That said if you need to manage 5,000 of them you start to debate the advantages of having the application in a virtual environment against the cost of buying 600 more servers. In most situations the difference is noise against the backdrop of security, maintenance, labor to manage a system -vs- the performance needed. Sadly many people think virtualization and/or containers are "Free" and make uninformed decisions that have real impacts on systems at scale later in the system's lifetime. In general if you need the flexibility or your application is usually 5% CPU 95% of the time you'd better off doing some form of virtualization to save labor and maximize resource utilization of systems you're deploying. That said I agree with many of the comments here it would be nice to see more head-to-heads of hypervisors but that's scary stuff for the vendors involved and puts them at serious risk in the arms race to save every penny we can in managing large farms of applications and the systems running them. Remember, Virtualization is just a technology like anything else it has it's benefits and brings with it resource usage and challenges. There is no "Free Lunch".. :-)
历次讨论
- 2014Hacker News 讨论帖163 分 · 18 评
同领域还值得读
I/O 比 CPU 更快——让我们划分资源并消除操作系统抽象
I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions
指出在现代服务器硬件上 I/O 速度已接近甚至超过 CPU，传统操作系统抽象反而成为瓶颈。
系统与数据库2019 · 817 分 · 277 评进阶 · 约 30 分钟编辑精选使用 io_uring 的高效 I/O
Efficient IO with io_uring
由 Linux 内核开发者 Jens Axboe 撰写，介绍 Linux 新的异步 I/O 接口 io_uring。
系统与数据库2019 · 426 分 · 96 评进阶 · 约 45 分钟编辑精选线程对大多数用途而言是个坏主意（1995）
Threads Are a Bad Idea for Most Purposes (1995)
这篇 1995 年的文章由 John Ousterhout 撰写，讨论为什么线程对大多数用途而言是坏主意，并主张事件驱动编程通常更简单、更高效。
系统与数据库2017 · 423 分 · 382 评进阶 · 约 25 分钟编辑精选学习型索引结构之论证
The Case for Learned Index Structures
提出把索引看作一种"模型"：既然 B-tree 等结构本质上是在学习数据的分布，就可以用机器学习模型（如回归模型、神经网络）来近似键到位置的映射，从而替代 B-tree、哈希索引…
系统与数据库2017 · 398 分 · 66 评 · 引用 25进阶 · 约 60 分钟编辑精选FoundationDB：一个分布式、解耦的事务型键值存储
FoundationDB: A distributed, unbundled, transactional key value store
介绍 FoundationDB：一个分布式、解耦设计的事务性键值存储。
系统与数据库2021 · 283 分 · 99 评进阶 · 约 90 分钟编辑精选