系统与数据库2019kernel.dk编辑精选
使用 io_uring 的高效 I/O
Efficient IO with io_uring
进阶 · 约 45 分钟读完前置知识：Linux 系统编程、异步 I/O 基础、epoll/aio 了解
导读
这篇论文由 Linux 内核开发者 Jens Axboe 撰写，介绍 Linux 新的异步 I/O 接口 io_uring。它通过用户态与内核共享的提交队列（SQ）和完成队列（CQE）两个环形缓冲区，大幅减少系统调用和上下文切换，实现高效的异步 I/O，性能显著优于传统的 epoll 和 aio。
HN 关注点：io_uring 是 Linux I/O 子系统的重大革新，HN 读者关注其设计细节、性能表现以及在实际项目（如 libuv/Node.js）中的应用潜力。
后来怎么样
io_uring 已成为 Linux 高性能异步 I/O 的标准接口，被 liburing、数据库（如 PostgreSQL、ClickHouse）和 Node.js 等广泛采用，并催生了大量后续特性与安全研究。
HN 评论精选
这几条评论里：补充 2 亲历 2
亲历亲历者用 io_uring 原型替换 libuv 线程池，认为前景很好
我用 io_uring 做了一个替代 Node.js fs.read/write 的原型，希望最终能取代 libuv 当前的线程池实现。非常有前景！向 Jens 在这个接口上的辛勤工作和关于 API 的有益讨论致敬。
zbjornson查看英文原文
I made a prototype of a replacement for Node.js' fs.read/write using io_uring in hopes of it ultimately replacing libuv's current threadpool implementation. Definitely promising! Many props to Jens for his hard work on this and for his helpful discussions on the API. Benchmarks at [0] and prototype repo at [1] [0] https://github.com/libuv/libuv/issues/1947#issuecomment-485230126 [1] https://github.com/zbjornson/node-io-uring
补充补充资料：论文是 2019 年的，io_uring 此后发展迅速，Windows 也推出了类似 API
这是 2019 年的论文，之后 Linux 上 io_uring 接口有不少更新，可参考 liburing（事实上的标准库）的 changelog。仅 2022 年 io_uring 就收到约 500 次提交，今年新增了不少有趣特性（如 spawn、异步缓冲、零拷贝 send 等），Windows 也采用了类似的 API。
minraws查看英文原文
For people who are curious before/after reading it, this is from 2019, there are updates the io_uring interface in Linux, quite a few actually, here's a changelog of liburing kind of the defacto library for io_uring. https://github.com/axboe/liburing/compare/liburing-0.4...liburing-2.2 Actually, io_uring has received around 500 commits just in 2022. Here's a list of interesting feature additions(this year), - https://www.phoronix.com/news/Linux-LPC2022-io_uring_spawn - https://www.phoronix.com/news/Linux-520-XFS-uring-Async-Buff - https://www.phoronix.com/news/Linux-5.20-IO_uring-ZC-Send - https://www.phoronix.com/news/Linux-5.19-IO_uring io_uring like API was also adopted by Windows recently. Lots of fun.
亲历亲历者早在十多年前就实现了与 io_uring 类似的共享环形缓冲区机制
十多年前我为一个日志系统做过非常类似的事：用户态和内核共享环形缓冲区（用户态分配，用 get_user_pages 映射进内核），用户态线程直接写入数据、更新头指针，内核线程轮询变化，只在缓冲区接近高水位（如 85% 满）时才触发系统调用。缓冲区可以在进程死后继续存在，进程崩溃后内核线程仍能处理最后写入的消息。当时不少人不认可这种“写进缓冲区让内核自己取、靠轮询”的思路。
kazinator查看英文原文
I did something very similar for a logging system, more than ten years ago. Circular buffers were shared between user space and kernel. (Allocated by user space, mapped into kernel with get_user_pages.) User space threads just added messages to their circular buffer by copying in data, hitting a barrier and updating a head pointer. Like io_uring, I avoided too many system calls by having a kernel thread poll for changes. User space would only make a system call when the ring would near a "high water mark", like 85% full or whatever. The circular buffers could outlive the process which created them and attached them. A process could crash and go away; the kernel thread would still process the last messages written right up to the crash. Not everyone around was keen on the idea. What do you mean just stick it into a buffer and the kernel will pick it up. Polling? Bah.
补充指出 Genode 操作系统早已采用类似的双环形缓冲区异步 I/O 设计
有趣的是，Genode 也用两个环做异步批量传输 RPC（packet streams），只是叫提交队列和确认队列而非提交队列和完成队列。由于 Genode 应用通过与实现 block session 接口的服务通信来做 I/O，它本质上和 Linux 一样用两个环形缓冲区做异步 I/O。推荐阅读《Genode Foundations》前半部分，内容有趣且易读。
PyroLagus查看英文原文
Interestingly, Genode actually also uses two rings for async bulk transfer RPC (packet streams)[0], only the rings are called submit queue and acknowledgement queue rather than submission queue and completion queue. And since Genode applications do I/O by communicating with a service implementing the block session interface[1], which uses the aforementioned packet streams, Genode essentially uses two ring buffers for async I/O just like Linux does (well, I guess it's more the other way around). I can definitely recommend reading through the first half of Genode Foundations (the second half is just an API reference). It's really interesting and informative, and it's a surprisingly easy read as well. However, I'd wait till the 19.05 edition gets released, since that should be less than a month's wait if they manage to release it on time. [0] https://genode.org/documentation/genode-foundations-18-05.pdf (page 92) [1] (pages 94 & 116)
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2019Hacker News 讨论帖255 分 · 64 评
- 2022Hacker News 讨论帖20 分 · 0 评
- 2022Hacker News 讨论帖151 分 · 32 评
同领域还值得读
I/O 比 CPU 更快——让我们划分资源并消除操作系统抽象
I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions
指出在现代服务器硬件上 I/O 速度已接近甚至超过 CPU，传统操作系统抽象反而成为瓶颈。
系统与数据库2019 · 817 分 · 277 评进阶 · 约 30 分钟编辑精选线程对大多数用途而言是个坏主意（1995）
Threads Are a Bad Idea for Most Purposes (1995)
这篇 1995 年的文章由 John Ousterhout 撰写，讨论为什么线程对大多数用途而言是坏主意，并主张事件驱动编程通常更简单、更高效。
系统与数据库2017 · 423 分 · 382 评进阶 · 约 25 分钟编辑精选学习型索引结构之论证
The Case for Learned Index Structures
提出把索引看作一种"模型"：既然 B-tree 等结构本质上是在学习数据的分布，就可以用机器学习模型（如回归模型、神经网络）来近似键到位置的映射，从而替代 B-tree、哈希索引…
系统与数据库2017 · 398 分 · 66 评 · 引用 25进阶 · 约 60 分钟编辑精选FoundationDB：一个分布式、解耦的事务型键值存储
FoundationDB: A distributed, unbundled, transactional key value store
介绍 FoundationDB：一个分布式、解耦设计的事务性键值存储。
系统与数据库2021 · 283 分 · 99 评进阶 · 约 90 分钟编辑精选SQLite：过去、现在与未来
SQLite: Past, Present, and Future
这篇 VLDB 论文梳理 SQLite 的历史、当前架构与性能表现，并展望未来方向。
系统与数据库2022 · 282 分 · 86 评进阶 · 约 40 分钟编辑精选