系统与数据库2017research.fb.com
Scuba：深入 Facebook 的数据分析系统
Scuba: Diving into Data at Facebook
进阶 · 约 45 分钟读完前置知识：分布式系统基础、数据仓库/OLAP 概念、列式存储基础
导读
这篇论文介绍 Facebook 的实时数据分析系统 Scuba，用于对海量事件数据进行快速交互式查询和下钻分析。它强调秒级实时摄入、灵活的 ad-hoc 查询和易用的可视化界面，让工程师甚至非技术人员都能快速排查性能问题、发现异常。
HN 关注点：是因为 Scuba 是 Facebook 内部最受好评的工具之一，多位前员工现身说法，而 Honeycomb、Interana 等产品都源自它的设计思想。
后来怎么样
Scuba 成为 Facebook 内部核心分析工具，并直接催生了 Honeycomb.io、Interana 等受其启发的商业化可观测性产品，推动了实时事件分析这一产品类别的发展。
HN 评论精选
这几条评论里：补充 1 亲历 3
亲历前员工盛赞 Scuba 的易用性和实时性，并推荐其衍生产品
我在 Facebook 工作两年了，Scuba 是我最喜欢的内部工具之一。文章侧重实现，但使用时完全不用操心这些：灵活快速易用的 UI、极快的查询速度和实时数据（可以查到几秒前发生的事）才是关键。React 团队就用它收集开发时的 JS 警告，追踪最高频的报错信息。Honeycomb.io 和 Interana 都是前 Facebook 员工受 Scuba 启发做的产品，感兴趣的话可以试试。
sophiebits查看英文原文
I'm so happy to see this posted. I've been at Facebook for two years, and Scuba is hands-down one of my most favorite internal tools we have (and we have a lot of good ones). The article focuses a lot on the implementation, but thankfully you don't need to worry about that when using it. The flexible/quick/easy UI is what seals the deal for me, combined with the fast query times and realtime data -- you can use it to query things that happened literally seconds ago. On the React team we use it to collect all dev-time JS warnings from React that our engineers see so that we can easily track what the most high-firing messages are and how frequently they occur. I haven't tried them extensively, but honeycomb.io and Interana are both Scuba-inspired products by ex-Facebookers. If this tool sounds at all interesting to you, I'd definitely look into using them.
亲历亲历者称 Scuba 让 FB 的性能分析进入新纪元
我在 FB 待了 2006-2015，Scuba 彻底改变了 Facebook 分析性能和获取洞察的速度。基础设施团队常说 pre-Scuba 和 post-Scuba 两个时代。能对互联网规模的服务做实时聚合性能指标、按任意维度下钻排查，让我们对系统的检测和定位问题能力信心大增。UI 也很好用，经理、PM 甚至非技术业务角色都在用。这是我在 FB 最怀念的内部产品。
nocarrier查看英文原文
I was at FB from 2006-2015, and Scuba caused a phase change in how Facebook's performance was analyzed and how quickly insights were gained. For many of us in Infrastructure, we spoke of the pre-Scuba, post-Scuba eras. It was such a joy to have aggregated realtime performance metrics for our internet scale services and have the ability to drill down on just about any dimension when doing perf investigations. It gave us so much more confidence in our systems, both in our ability to detect issues, as well as pinpoint what was causing them. I'd also like to echo what spicyj said about the UI, it is very usable. It was common for managers, PMs, and even people in completely non-technical business roles to use Scuba. It was my favorite internal product at FB and the one I miss the most.
亲历可视化与概率分布分析帮助快速定位客户性能问题
前 FB 员工、现 Google 员工，非常怀念 Scuba，可视化功能尤其出色。在 Parse 时我通过把查询（日志中预计算）拆分成族并查看概率分布，诊断过客户非常复杂的性能问题。每当客户说问题是 XYZ，我就能发截图，把讨论拉回数据指向的方向。一些前 Parse 员工基于同样的论文做了 Honeycomb.io，希望再次把它加入工具箱。
inlined查看英文原文
Former Facebook employee and current Googler. I really really miss Scuba. The visualizations are what really cinched things for me. At Parse I was able to diagnose some seriously complex performance issues for customers by breaking splitting queries (precomputed in logs) into families and looking at probability distributions. It was amazing. Every time the customer told me XYZ was the problem I could send them a screenshot and refocus the conversation where the data sent us. Some ex Parse employees left FB to build a visualization tool based on the same white papers (honeycomb.io). I'm really hoping to add that to my tool belt again.
补充Interana 创始人自荐其受 Scuba 启发的商业化产品
另有一篇相关博客文章。我的 YC 公司 Interana 借鉴了这套系统的很多经验，为 Reddit、Sonos、Comcast、Bing 这类公司提供更好的方案，欢迎注册试用。
liorabraham查看英文原文
There's also a blog post about it here https://www.facebook.com/notes/facebook-engineering/under-the-hood-data-diving-with-scuba/10150599692628920/ My YC company https://www.interana.com took a lot of lessons from this and is doing something that I think is even better for cos like Reddit, Sonos, Comcast, Bing. You can sign up if you'd like a demo :)
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2017Hacker News 讨论帖206 分 · 41 评
同领域还值得读
I/O 比 CPU 更快——让我们划分资源并消除操作系统抽象
I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions
指出在现代服务器硬件上 I/O 速度已接近甚至超过 CPU，传统操作系统抽象反而成为瓶颈。
系统与数据库2019 · 817 分 · 277 评进阶 · 约 30 分钟编辑精选使用 io_uring 的高效 I/O
Efficient IO with io_uring
由 Linux 内核开发者 Jens Axboe 撰写，介绍 Linux 新的异步 I/O 接口 io_uring。
系统与数据库2019 · 426 分 · 96 评进阶 · 约 45 分钟编辑精选线程对大多数用途而言是个坏主意（1995）
Threads Are a Bad Idea for Most Purposes (1995)
这篇 1995 年的文章由 John Ousterhout 撰写，讨论为什么线程对大多数用途而言是坏主意，并主张事件驱动编程通常更简单、更高效。
系统与数据库2017 · 423 分 · 382 评进阶 · 约 25 分钟编辑精选学习型索引结构之论证
The Case for Learned Index Structures
提出把索引看作一种"模型"：既然 B-tree 等结构本质上是在学习数据的分布，就可以用机器学习模型（如回归模型、神经网络）来近似键到位置的映射，从而替代 B-tree、哈希索引…
系统与数据库2017 · 398 分 · 66 评 · 引用 25进阶 · 约 60 分钟编辑精选FoundationDB：一个分布式、解耦的事务型键值存储
FoundationDB: A distributed, unbundled, transactional key value store
介绍 FoundationDB：一个分布式、解耦设计的事务性键值存储。
系统与数据库2021 · 283 分 · 99 评进阶 · 约 90 分钟编辑精选