When your website stops working at 3 AM, you need to answer one question fast: Is it my code or is a big cloud provider having problems? Omri Sass from Datadog explains updog.ai, a tool that monitors whether major services like AWS, CloudFlare, and others are actually working. Instead of asking people to report problems like Down Detector does, updog uses real data from thousands of computers to detect when services go down. Omri shares why this took 6 years to build, how they process massive amounts of data with machine learning, and why cloud providers have been strangely upset about these tools existing.

About Omri:

Omri Sass is a Director of Product Management at Datadog, where he leads and supports a team of 25+ product managers driving initiatives across Bits AI SRE, Data Observability, Service Management, and most recently, the launch of updog.ai. Outside of work, Omri is an avid sci-fi reader, a dedicated yoga practitioner, and happily outmatched by his cat.

Show Highlights:

(02:12) What is Updog and How Does It Work

(03:38) Why Knowing If It's a Global Problem Matters

(04:01) The Problem With Testing Every Endpoint Yourself

(05:52) How Datadog Discovered EC2 Outages From Their Own Systems

(10:38) When AWS Regions Go Down and Cascade Failures

(13:13) What Happens When Services Rebuild Completely
(16:29) The Most Important Learning During a 3 AM Incident
(20:11) Why This Took So Long to Build
(23:40) When Datadog Going Down Isn't Critical Path
(25:22) How They Picked Which AWS Services to Monitor
(27:07) What Comes Next for Updog
(30:11) Where to Find Omri and Updog

Links:

Datadog: datadoghq.com

Omir’s LinkedIn: https://www.linkedin.com/in/omri-sass-65632a14/

Sponsored by:
duckbillhq.com

Hunter Leath, CEO of Archil, spent 8 years building Amazon's EFS file storage system, learning exactly why making cloud storage act like a hard drive always fails. Old programs need hard drives, but cloud storage doesn't work like hard drives—a problem that's existed for 20 years.

Now Hunter's building Archil, which puts super-fast storage between programs and S3 so they can finally work together. Your programs think they're talking to a regular disk while your data lives safely in the cloud.

Hunter explains how they're doing what others couldn't, why it costs less than Amazon's own solutions, and why file systems suddenly matter again in the AI era.

Show Highlights:

(01:37) What Archil Does and Why It Exists

(02:26) Why Mounting S3 as a File System Has Always Failed

(03:07) What Building EFS Taught Hunter

(06:55) Using Fast SSDs as a Cache Layer for S3

(09:45) Attaching Archil to Your Existing S3 Buckets

(15:08) Why Archil Costs Less Than EBS When You Do the Math

(17:56) What Happens If Amazon Builds This Feature

(19:20) Competing With EBS Performance on GP3 Volumes

(21:43) Raising $6.7 Million Without an AI Pitch

(23:46) What Customers Get Wrong About Archil

(28:07) Accessing Data Stored in Glacier Deep Archive

(29:24) The Plan to Get Into the Linux Kernel

(30:51) Where to Find Hunter

About Hunter Leath:

Hunter is the founder and CEO of Archil, which transforms S3 buckets into infinite, local file systems that provide instant access to massive data sets. Prior to Archill, Hunter spent the last ten years in the cloud storage industry, including 8 years building Amazon's Elastic File System product and one year on Netflix's core storage team.

Links:
Hunter Leath on LinkedIn: https://www.linkedin.com/in/hleath/

Hunter Leath on X: https://x.com/jhleath/

Archil’s Website: https://archil.com

Sponsored by:
duckbillhq.com

When AWS has a major outage, what actually happens behind the scenes? Ben Hartshorne, a principal engineer at Honeycomb, joins Corey Quinn to discuss a recent AWS outage and how they kept customer data safe even when their systems couldn't fully work. Ben explains why building services that expect things to break is the only way to survive these outages. Ben also shares how Honeycomb used its own tools to cut their AWS Lambda costs in half by tracking five different things in a spreadsheet and making small changes to all of them.

About Ben Hartshorne:

Ben has spent much of his career setting up monitoring systems for startups and now is thrilled to help the industry see a better way. He is always eager to find the right graph to understand a service and will look for every excuse to include a whiteboard in the discussion.

Show highlights:

(02:41)Two Stories About Cost Optimization

(04:20) Cutting Lambda Costs by 50%

(08:01) Surviving the AWS Outage

(09:20) Preserving Customer Data During the Outage

(13:08) Should You Leave AWS After an Outage?

(15:09) Multi-Region Costs 10x More

(18:10) Vendor Dependencies

(22:06) How LaunchDarkly's SDK Handles Outages

(24:40) Rate Limiting Yourself

(29:00) How Much Instrumentation Is Too Much?

(34:28) Where to Find Ben

Links:

Linkedin: https://www.linkedin.com/in/benhartshorne/

GitHub: https://github.com/maplebed

Sponsored by:
duckbillhq.com