Cover of The Infra Pod

The Infra Pod

The Infra Pod

By: The Infra Pod
Listen for free

Season's Savings | 0.99 €/month for 3 months

5.99 €/month thereafter - terms apply. Cancel monthly.
Summary
The Infra Pod brings you insightful and thought-provoking discussions on the world of infrastructure software. This podcast is started by two engineers, Ian Livingstone (tech advisor for Snyk) and Tim Chen (General Partner at Essence VC), team up with a rotating cast of guests to dive deep into the latest trends and hot topics in the software infrastructure space.The Infra Pod
Episodes
  • AI Is the Fourth Application Architecture! Chat with Dave McJannet (ex-CEO of Hashicorp, CEO of Dome Systems)
    6 Oct 2026

    Dave McJannet — CEO of Dome Systems, former CEO of HashiCorp, and a veteran of Cloud Foundry, Hortonworks, and GitHub — joins Tim (Essence) and Ian Livingston (Keycard) to argue that AI isn't a feature wave, it's the fourth application architecture after mainframe, client-server, and cloud.


    Dave walks the stack layer by layer to separate what endures from what gets layered over: compute shifting from CPU to GPU and onto an inferencing layer, messaging (Kafka vs. Temporal) becoming more critical, not less, APM racing to monitor a new substrate, and a data stack that probably doesn't get rebuilt at all. He makes the case that agent governance is a DevOps problem, not a security one — the platform-engineering team that can't answer "what will this thing do once deployed?" is what's actually blocking enterprise agents today, exactly like security teams stalled cloud growth in 2015 before the cloud 2.0 playbook unlocked it.


    Plus: why he thinks the open-source business model is broken in a world where an agent can reconstruct your repo overnight, what replaces it as a moat, why heterogeneity of models is inevitable (and why India got there first), and a spicy take that vertical SaaS and the incumbents are far more defensible than the Valley believes.


    - 0:00 — Intro & AI as the fourth application architecture

    - 12:39 — Why Dome Systems: agent governance as a DevOps problem

    - 22:56 — Enterprise adoption, heterogeneity, and the death of open source as a moat

    - 35:49 — The Spicy Future: hot takes and closing thoughts

    Show more Show less
    45 min
  • 1,000 tokens per second: the case against predicting one word at a time (Kumar, VP of Engineering at Inception)
    24 Aug 2026

    What if the entire LLM industry has been solving language generation the slow way — one token at a time?


    In this episode of The Infra Pod, hosts Tim Chen (GP at Essence VC) and Ian Livingstone (CEO of Keycard) sit down with Kumar, VP of Engineering at Inception, to unpack Mercury 2, the company's diffusion-based language model, and make the case for a fundamentally different way to generate text and code.


    Kumar breaks down the core mechanical difference: where a GPT-style transformer predicts the next token one pass at a time, a diffusion model predicts a whole batch of upcoming tokens at once and iteratively denoises them in parallel, freezing the easy ones early and spending extra compute only on the hard ones. That approach — borrowed from image generation but re-engineered for text, where output length isn't known in advance and streaming is a hard requirement — yields roughly a 10x speedup and 3-5x cost efficiency by simply doing fewer forward passes. Mercury 2 hits 1,000 tokens/second on commodity NVIDIA hardware and now benchmarks competitively against cost-optimized models like Claude Haiku, Gemini Flash, and GPT-mini, though Kumar is candid that no diffusion model — Inception's included — has yet reached Sonnet or frontier-tier intelligence.


    The conversation moves from algorithm to product: why Inception keeps its API OpenAI-compatible (Kumar's electric-car analogy — same interface, very different feel under the hood), why most agentic sub-tasks don't need frontier intelligence at all, and why voice and search are the workloads where sub-second latency stops being a nice-to-have and becomes existential. Kumar closes with a genuinely spicy take!


    [00:00] Guest introduction: Kumar, VP of Engineering at Inception Labs

    [01:31] Diffusion vs. autoregressive LLMs: what's actually different under the hood

    [07:00] Why isn't diffusion the default already? Trade-offs and diffusion's late start on text

    [09:05] From Mercury 1 to Mercury 2: the road to enterprise-readiness

    [11:26] Open source diffusion models, ICML's best paper, and Inception's head start

    [16:15] Under the hood: how Inception hits 1,000 tokens/sec without sacrificing latency

    [20:52] Data strategy: what training a diffusion model actually requires

    [22:51] Does diffusion change how you build agents and products on top of it?

    [28:00] Mercury 2 benchmarked against cost-optimized frontier models

    [29:23] The model routing problem — and why it may already be mostly solved

    [37:47] Spicy Future: AGI


    Show more Show less
    44 min
  • What happens to your service mesh when the workloads running on it aren't written by humans? (Chat with William at Buoyant)
    20 Jul 2026

    In this episode of The Infra Pod, hosts Tim Chen (GP at Essence VC) and Ian Livingstone (CEO of Keycard) sit down with William Morgan, co-founder and CEO of Buoyant and creator of Linkerd, to explore how AI agents are reshaping the infrastructure layer — from service security to inference routing.


    Will traces Linkerd's origins to Twitter's 2014 migration from a monolithic Rails app to distributed microservices — the moment function calls became network calls that could actually fail. A decade later, that same communication layer is under pressure again. Non-deterministic agents make MCP and A2A calls over L7 protocols, and for the first time, fine-grained access control isn't optional: an agent will eventually find and call every reachable endpoint, including the one that deletes your database.


    The conversation covers the real pressure AI coding tools are already placing on platform teams — deploy cadences going from tens to potentially thousands per day — and what running inference inside Kubernetes actually means for proxies. Will breaks down why KV cache-aware routing is a 100x performance lever, why the modern inference proxy looks less like Envoy and more like a Makefile, and shares his spicy take on where compute is heading: in-cluster inference becomes the default, with frontier models reserved only for tasks that genuinely need godlike intelligence.


    [00:00] Guest introductions and Buoyant founding story

    [03:00] Linkerd's origin: solving Twitter's monolith-to-microservices migration

    [07:30] How AI is (and isn't) changing Linkerd today

    [11:00] MCP, A2A, and agents as L7 traffic in your cluster

    [14:30] Why agents make endpoint-level access control non-optional

    [18:00] AI as amplifier: what 10–1000x more deploys means for platform teams

    [21:30] Running inference in Kubernetes: a pathological workload

    [24:45] KV cache-aware routing and the 100x performance gap

    [27:00] The proxy/gateway landscape: grad students vs. premature standardization

    [29:30] The inference proxy is a Makefile now, not Envoy

    [31:00] Prompt injection, sandboxing, and the security problems with no clean answer

    [33:00] Spicy Future: in-cluster inference becomes the default

    Show more Show less
    34 min
adbl_web_anon_alc_button_suppression_t1
No reviews yet