What do you do when an experiment tanks conversion rate by 8% in less than a week?In this episode, Gerda and Rommil break down real CRO and experimentation scenarios submitted by the community, including failed tests, executive pressure, guardrail metrics, knowledge management, developer friction, and why experimentation teams always seem to end up back in spreadsheets.Check out: http://katsed.com/We also discuss:

* Root cause analysis for failed experiments

* How to calm leadership during testing disasters

* Guardrail metrics and control limits

* Why experimentation knowledge banks become messy

* Airtable vs spreadsheets vs enterprise tools

* Why developers and designers struggle with experimentation culture

* How to onboard technical teams into testing workflows

A practical conversation for experimentation leads, growth teams, product managers, CRO specialists, and anyone building an experimentation culture.Chapters00:00 Burger Menu Debate: Left vs Right01:28 Mystery Meat Navigation & UX History03:03 Why Modern Phones Break UX05:08 Community CRO Scenarios Begin06:18 Experiment Tanks Conversion by 8%07:33 Root Cause Analysis & Leadership Pressure09:17 QA, Tracking Errors & Failed Tests10:03 Building a CRO Analysis Tool12:18 Probabilistic Root Cause Detection14:26 Managing Panic During Bad Experiments15:21 Guardrail Metrics Explained16:40 Using Control Limits in Experimentation18:18 Building an Experimentation Knowledge Bank20:05 Why You Don’t Need AI for Everything21:08 Why Process Changes Create Admin Pain23:08 Migrating Between Experimentation Tools24:31 Why Teams Always Go Back to Spreadsheets26:50 CRO Teams vs Developers & Designers29:09 Why Early Experiments Should Be Simple30:41 Why Developers Hate “Throwaway Code”32:16 Designers vs Experiment Constraints33:02 Outro & Next Community Questions

Most teams say they “do research”… but very few actually do it well.In this episode, we break down what real customer research looks like in CRO and growth. From analyzing 200+ survey responses manually, to why AI still struggles with qualitative data, to the politics that slow everything down.We also get into:Why AI fails at nuanced research (and what it misses)How to actually analyze open-ended survey responsesThe biggest mistakes teams make with surveysWhy companies avoid talking to customers (and why that’s a red flag)How research becomes a political nightmare in larger orgsWhen you should sample vs analyze everythingHow research fuels better marketing and personalizationAnd yes… we somehow end up talking about billion dollars and laundry.If you’re in CRO, growth, product, or marketing, this is the stuff people don’t usually say out loud.👇 Timestamps00:00 Why content actually drives opportunities03:15 What “research” really means04:00 How many responses you actually need06:00 Why AI fails at qualitative analysis07:30 When to sample vs analyze everything09:00 Using research for marketing copy10:00 Why not talking to customers is a red flag11:00 Survey politics and internal resistance13:30 Bad survey questions and bias16:00 Personalization vs CRO (what’s the difference?)22:30 Personalizing outreach properly26:30 Why research projects spiral out of control29:00 How to actually make research a habit31:00 How to ship research faster (timeboxing)34:00 “Business jiu-jitsu” and fake urgency35:30 Final thoughts (and laundry)

In this episode, we dive into the realities of CRO (Conversion Rate Optimization) today — from AI anxiety to experimentation culture and how teams should actually be structured.

We start with a surprisingly philosophical take on photography and overthinking, then quickly shift into what’s really happening in the CRO and AI space right now.

We discuss:

* Why AI might be making everything look the same

* The growing fear (and noise) around AI in CRO

* How to structure CRO teams: centralized vs hybrid vs embedded

* Why experimentation still struggles with a “nice-to-have” perception

* What actually builds a strong experimentation culture

* How individual contributors can influence culture without authority

* And how to position CRO as a true growth engine, not just testing

If you work in growth, product, or experimentation, this is a candid look at what’s changing and what actually matters.

Chapters

0:00 – Photography, overthinking, and creative process1:26 – Back to CRO: what’s happening in the industry2:07 – AI concerns and job market anxiety2:45 – The “everything looks the same” AI problem4:00 – Too many AI tools, no real differentiation5:38 – What people are really asking about AI6:23 – AI fear vs reality (and layoffs)7:08 – Fear-mongering vs signal8:00 – Fear and decision-making9:00 – CRO team structures: centralized, hybrid, embedded9:31 – Why centralized works early10:08 – When to move to hybrid10:29 – Why embedded is hard11:04 – The role of leadership buy-in12:13 – Why experimentation gets cut first13:00 – CRO’s “nice-to-have” branding problem14:00 – Why CRO is easy to underestimate15:45 – Future-proofing: combine CRO with other skills16:54 – Building experimentation culture (the “laundry” analogy)18:19 – How to destroy experimentation culture19:23 – Why authority and buy-in matter20:25 – How ICs can influence culture21:34 – Reframing experimentation as empowerment23:09 – Becoming a catalyst across teams24:46 – Confidence and presence in CRO roles25:59 – CRO vs Growth: what’s the difference?27:24 – Where CRO fits inside growth teams28:02 – Moving CRO upstream into systems thinking29:00 – Closing thoughts