网络与分布式2020arxiv.org
使用 BSD socket 实现 10 Gb/s 可靠 UDP 传输
Achieving reliable UDP transmission at 10 Gb/s using BSD socket
进阶 · 约 30 分钟读完前置知识：UDP/TCP 协议基础、Linux 套接字编程、网络性能调优
导读
这篇论文研究在探测器读出系统这类点对点场景中，如何通过 BSD socket 实现 10 Gb/s 的 UDP 数据传输，并总结吞吐量优化手段，包括 MTU、包大小、Linux 内核参数调优、线程亲和性、核心局部性和高效定时器。论文还讨论了可靠传输的必要性：某些探测器读出场景下保证送达并非必需，偶尔重传可能不值得增加复杂度。
HN 关注点：高性能网络传输的工程实践：10Gb/s 下如何优化 UDP 吞吐量，以及“可靠 UDP”这一提法是否名副其实。讨论焦点在于实际场景是否真的需要可靠性，以及测试条件是否具有代表性。
HN 评论精选
这几条评论里：质疑 1 补充 3
补充补充了 Cloudflare、Netflix、BBC 等相关高性能网络资料。
(摘要) 吞吐量优化包括：MTU、包大小、调优 Linux 内核参数、线程亲和性、核心局部性和高效定时器。 Cloudflare 的 majke 分享过一系列类似主题的文章 [0][1][2]（重点是实现线速、更高的每秒包数和更低的延迟，而非吞吐量），我觉得非常有帮助，尤其是它们非常详尽 [3]。 说到吞吐量，drewg123 写过一篇关于 Netflix 如何用 FreeBSD 网络栈实现 100gbps 的文章 [4]，这里是 BBC 关于他们如何绕过 Linux 网络栈实现 [5] 的文章。
ignoramous查看英文原文
> (abstract) Optimizations for throughput are: MTU, packet sizes, tuning Linux kernel parameters, thread affinity, core locality and efficient timers. Cloudflare's u/majke shared a series of articles on a similar topic [0][1][2] (with focus on achieving line-rate with higher packets-per-second and lower latency instead of throughput) that I found super helpful especially since they are so very thorough [3]. Speaking of throughput, u/drewg123 wrote an article on how Netflix does 100gbps with FreeBSD's network stack [4] and here's BBC on how they do so by bypassing Linux's network stack [5]. --- [0] https://news.ycombinator.com/item?id=10763323 [1] https://news.ycombinator.com/item?id=12404137 [2] https://news.ycombinator.com/item?id=17063816 [3] https://news.ycombinator.com/item?id=12408672 [4] https://news.ycombinator.com/item?id=15367421 [5] https://news.ycombinator.com/item?id=16986100
补充提到重复发送相同 UDP 包可大幅提高到达率的技巧，并追问其现状。
优化互联网上的 UDP 传输是个有趣的话题。我记得前段时间读过一篇论文，表明如果你通过互联网发送两个连续、数据完全相同的 UDP 包，至少其中一个会以相当高的成功率到达目的地（大约 99.99%）。我想知道这在当前的互联网基础设施下是否仍然有效，以及这个技巧是否还在实时流媒体协议中使用。
snisarenko查看英文原文
Optimizing UDP transmission over internet is an interesting topic. I remember reading a paper a while ago that showed that if you send two consecutive UDP packets with exact same data over the internet, at least 1 of them will arrive to the destination at pretty high success rate (something like 99.99%) I wonder if this still works with current internet infrastructure, and if this trick is still used in real-time streaming protocols.
补充提问能否用 TCP 增大包大小来降低开销、同时保持可靠性。
你能用 TCP 做类似的事情吗？增大包大小，使“TCP 开销”相比 64 字节载荷减少，同时保留 TCP 相对 UDP 更高的可靠性？
exdsq查看英文原文
Can you do something similar with TCP and increase the packet size such that the "TCP Overhead" is reduced compared to 64 byte payloads but with the increased reliability over UDP?
质疑质疑论文标题与内容不符：场景是点对点且未必需要可靠传输，测试条件（2 米无源 DAC）也不代表真实光传输。
论文描述的场景里，网络只有发送方和接收方（可选一个交换机），拥塞只发生在两端；读出系统以接近恒定速率产生数据，用 TCP 时接收方拥塞会导致发送方降速。按这个说法，更好的标题应该是“增加缓冲区或优化应用系统调用以接收 10 GB/s 数据”，因为它跟实现可靠 UDP 传输无关——他们似乎甚至不需要可靠性：“在一个探测器原型中，我们因阈值抑制丢弃了约 24% 的数据，所以花额外时间做偶尔重传可能不值得增加复杂性。”另外，实际测试用的是 2 米无源 DAC，误码率约 10^-18（约每 100 PB 错 1 比特），而现实中的光传输即使带 FEC 也常有约 10^-12 的误码率，测试场景的代表性值得商榷。
zamadatix查看英文原文
"In a readout system such as ours the network only consists of a data sender and a data receiver with an optional switch connecting them. Thus the only places where congestion occurs are at the sender or receiver. The readout system will typically produce data at near constant rates during measurements so congestion at the receiver will result in reduced data rates by the transmitter when using TCP." At that point a better paper title would have been "Increasing buffers or optimizing application syscalls to receive 10 GB/s of data" as it has nothing to do with achieving reliable UDP transmission, which it doesn't even seem they needed: "For some detector readout it is not even evident that guaranteed delivery is necessary. In one detector prototype we discarded around 24% of the data due to threshold suppression, so spending extra time making an occasional retransmission may not be worth the added complexity" As far as actual reliable UDP testing at high speeds one might also want to consider the test scenario as not all Ethernet connections are equal. The 2 meter passive DACs used in this probably achieve ~10^-18 bit error rate (BER) or 1 bit error in every ~100 petabytes transferred. On the other hand go optical even with forward error correction (FEC) it's not uncommon to expect transmission loss in the real world. E.g. looking at something a little more current https://blogs.cisco.com/sp/transforming-enterprise-applications-with-25g-ethernet-smf is happy to call 10^-12 with FEC "traditionally considered to be 'error free'" which would have likely resulted in lost packets even in this 400 GB transfer test (though again they were fine with up to 24% loss in some cases so I don't think they were worried about reliable as much as reading the paper title would suggest). Generally if you have any of these: 1) unknown congestion 2) unknown speed 3) unknown tolerance for error You'll have to do something that eats CPU time and massive amounts of buffers for reliability. If you need the best reliability you can get but you don't have the luxury of retransmitting for whatever reason then as much error correction in the upper level protocol as you can afford from a CPU perspective is your best bet. If you want to see a modern take on achieving reliable transmission over UDP check out HTTP/3.
历次讨论
- 2020Hacker News 讨论帖167 分 · 42 评
同领域还值得读
Paxos vs. Raft：分布式共识是否已达成共识？
Paxos vs. Raft: Have we reached consensus on distributed consensus?
重新审视了 Paxos 与 Raft 两种分布式共识算法之间的差异，试图在一个统一的框架下比较它们，并论证二者的相似程度远高于通常的认知，许多所谓的差别只是表述层面的。
网络与分布式2020 · 343 分 · 65 评 · 引用 10进阶 · 约 28 分钟编辑精选Stellar 协议：互联网级共识的联邦模型
Stellar Protocol: A Federated Model for Internet-Level Consensus (2016)
提出 Stellar Consensus Protocol（SCP），一种面向互联网级共识的联邦拜占庭协议模型。
网络与分布式2018 · 301 分 · 61 评专业 · 约 90 分钟编辑精选拜占庭将军问题（1982）
The Byzantine Generals Problem (1982)
这篇 1982 年论文提出并形式化了拜占庭将军问题：在分布式系统中，部分节点可能叛变或故障、通信不可完全信任时，忠诚节点如何达成一致行动。
网络与分布式2018 · 290 分 · 77 评进阶 · 约 75 分钟编辑精选Google 变差了吗？对搜索引擎中 SEO 垃圾的纵向调查
Is Google Getting Worse? A Longitudinal Investigation of SEO Spam in Search
对搜索引擎结果中的 SEO 垃圾内容做了纵向追踪：作者团队在一段时间内持续采集 Google、Bing、DuckDuckGo 等引擎针对商品评测类查询的结果，用分类器识别联盟营销…
网络与分布式2024 · 276 分 · 274 评入门 · 约 45 分钟编辑精选使 CRDT 能够容忍拜占庭故障
Making CRDTs Byzantine Fault Tolerant
Martin Kleppmann 等人的论文研究如何让 CRDT（无冲突复制数据类型）抵御拜占庭故障节点，即部分参与者恶意篡改数据时仍能保证一致性。
网络与分布式2022 · 266 分 · 40 评进阶 · 约 90 分钟编辑精选