Karpenter has achieved GA and is disrupted the autoscaling game, with data pointing to accelerated adoption.

Today, Josh Cypher, DevOps leader at Sonos, joins us to talk about some unexpected byproducts of adopting Karpenter at Sonos. Josh dives into his favorite features and efficiency gains, from node consolidation to better disruption controls.

The public cloud bill is a massive operational expense for tech organizations, yet tracking the success of optimization efforts often frustrates engineers. Josh and James explore these challenges and how to address them effectively.

As big-time Spot adopters, Sonos has unlocked impressive savings (50%?!) by focusing on high-impact, low-overhead strategies. Josh explains how visibility brought them quick wins and paved the way for further optimization across Sonos’s infrastructure.

Plus, Josh and James preview what they’re looking forward to at KubeCon, including key conversations on Kubernetes, AI, and cloud sustainability.

Dr. Haoran Qiu, a fresh PhD from the University of Illinois Urbana-Champaign, joins our host James Wilson, VP of Engineering at nOps. They’re diving into multidimensional autoscaling, an area in which Haoran’s pioneering research is making waves in the Kubernetes community.

Some workloads work better with Horizontal Pod Autoscaler (HPA), others with Vertical Pod Autoscaler (VPA). Running them together can create conflicts, but using only one limits efficiency gains. A Multidimensional Pod Autoscaler solves this dilemma by combining the benefits of both VPA and HPA to dynamically adjust both the number and size of pods.

But is MPA poised to redefine resource optimization? What problems does it solve, and what fresh complexities are involved in its implementation?

Haoran and James dig into these questions while debating traditional heuristic versus Machine Learning approaches, industry versus academia, and other hot topics in Kubernetes.

Listen now to discover if MPA is the holy grail of cloud optimization as we discuss the evolution of autoscaling technologies and their impact on cost, sustainability, and developer experience.

Chapters:

0:00 - 2:20: Haoran Chu and the state of cloud resource management

2:20-6:00: Historical evolution of autoscaling

6:01 - 10:45: HPA, VPA and Multidimensional Autoscaling

10:46 - 18:50: Challenges of MPA: heuristics versus machine learning

18:51 - 24:20: How to quantify excess capacity?

24:21 - 32:16: The state of ML in autoscaling

32:16 - 37:37: Operationalizing ML in production environments
37:37 - 42:01: The near-term future of autoscaling

Tech thought leader and host of the Kubernetes Unpacked podcast Kristina Devochko joins nCast today to talk all things cloud cost optimization, Kubernetes and green tech.

We start by talking about the fact that many companies aren’t even using HALF of their compute resources. But does slashing your AWS bill necessarily mean that you’re saving the plant? We delve into cost optimization and how it aligns (or not) with sustainability.

Kristina shares her insights on measuring your cloud carbon footprint and the tools you need (KEDA, Karpenter, Kepler) to increase cloud sustainability. We discuss key practical ways to get started cutting unnecessary cloud waste, from eliminating orphaned resources to scheduling during off hours.

Plus, we're revealing how nOps has managed to run our production on Spot instances — talk about recycling!

0:00 - 1:09: Introduction

1:10 - 4:20: Sustainability at Kubecon Europe and other recent events

4:21 - 9:31: Is cost optimization the same as sustainability?

9:32 - 12:53: Green data centers and your carbon footprint

12:54 - 15:21: Portability and the downsides of over-committing to pricing plans

15:22 - 19:51: Measuring your organization’s cloud sustainability

19:53 - 26:51: KEDA, Karpenter, Kepler and the tools you need

26:52 - 31:12: Leveraging available Spot capacity and choosing instances

31:13 - 37:15: Running production environments on Spot

37:15 - 44:46: Continual rightsizing and automated tools

44:47 - 48:18: Carbon-efficient Karpenter scaling

Show notes

• GitHub issue for proposal of carbon-efficient design to Karpenter that needs some community support

• Kepler project

• Carbon-aware KEDA operator

• Cloud Carbon Footprint open source tool

• BoaviztAPI open source API for environmental impacts of ICT

• APIs that provide electricity data, data on carbon emissions and electricity sources: https://app.electricitymaps.com and https://watttime.org

• CNCF TAG Environmental Sustainability

• Contact Kristina Devochko

• Kristina Devochko’s Tech blog
  

Today we’re joined by Wade Piehl, Senior FinOps Success Manager on the Optics Team at AWS, to discuss all things On Demand, Reserved, and Spot.

Good purchasing decisions have an enormous impact on your bottom line — but how do you know what and how much to buy? We walk through the step-by-step decision-making process and the most common pitfalls that can cost you.

Wade shares all the practical advice you need to decide between Reserved Instances vs. Savings Plans. Find out how the equation changes from compute spend to database, storage and networking. We’ll also dig into strategies like layering purchase plans, leveraging Spot, and other methods of maximizing your savings.

Today we’re joined by Savanna Jensen, Senior FinOps Success Manager on the Optics Team at AWS, to discuss how to implement a Cloud FinOps automation strategy. The less amount of “people time” you can dedicate to cloud management and the more automation you can bake into the system, the easier it will be — but what are the right tools to use?

We start out by tapping Savanna’s insider knowledge on the latest and greatest AWS Cost Management tools. Get the latest on the shiny new updates to Cost Explorer that just launched. Plus, pro tips on the best filters and features to use for various use cases when it comes to the Cost and Usage Report (CUR), Cost Explorer, and QuickSight.

We dive into Engineering vs. FinOps perspectives and frustrations. How can you orchestrate a culture where cost optimization feels motivating rather than punitive to Engineers (recognition, career rewards, gamification…)? How do you troubleshoot if you’re a FinOps leader seeing zero traction on your initiatives?

Hear real-world battle stories about organizations tackling cost management challenges — and the takeaways for achieving true visibility and control over cloud costs. And stay tuned next week for Part 2 with the AWS Optics team.

In Episode 9, we’re joined by AI expert Marcos Heidemann to pull the curtain back on GenAI and whether the hype means we’re at the cusp of a massive transformation. Will it automate away our jobs in a dystopian future, or is it just glorified autocomplete?

Our panelists do a technical dive into new technologies like ChatGPT and LangChain. How does GenAI think, and what does that mean for building commercial products on it?

We’ll give you a sneak peak into nOps’s firsthand experience using GenAI to solve real-world cost optimization problems in novel ways that have never been tried before — and what we’re currently in the process of rolling out.

The conversation wraps up with hot takes on the current state of AI technology and where it’s going. What do ChatGPT 5, 6, 7, and their competitors look like? What’s the moonshot for GenAI over the next 2-5 years?

Listen now to find out.

In Episode 8, we’re joined by AWS Partner Solutions Architects Andrew Park and Mike McDonald to discuss the complexities and cost of running today’s ML and AI workloads on the cloud.

From anecdotes of the bad old days before container orchestration, our panelists take you to the present challenge of how to simplify efficient infrastructure operation — with the aim of freeing up Data Scientists and Engineers to focus on building and innovating.

Our panelists discuss the merits, pitfalls, and potential of various cost-optimizing tools and approaches (Ray, Karpenter, Spot, timeslicing) — key to addressing the demand for the expensive computing power generated by ML and AI models at scale.

Watch the full episode for:

• The lowdown on AWS Bedrock and where it fits into the current stack of the latest AWS ML and AI offerings — how it works, use cases, the access it grants to new generative AI models

• How Karpenter can make your life easy and save you SO much money (especially if you set-it-and-forget-it with nKS)

• And hot takes on the controversial question: is ECS dead?!

We’re thrilled to host Marit Hughes, a Specialist Master for Government and Public Sector at Deloitte. After debating the best FinOps conferences to attend this year, we tackle the question: with billions and billions of dollars being spent on cloud resources, why is it so hard to make cost optimization actually happen?

And as we navigate the unique maze of public sector cloud optimization, get the inside scoop on why it's a different beast from commercial and the conversations taking place behind the scenes.

We discuss the realities of engineering life in the trenches — from lack of tooling and visibility to 18-hour days flooded by JIRA tickets. (Plus, bonkers things we’ve seen in bills). Whether you’re in public sector or private, how do you make cost management a lot less painful for engineers?

The other side of the question is where and how FinOps practitioners should insert themselves to execute and achieve quantifiable results. (Hint: telling engineers their baby is ugly isn’t going to help). Marit reveals the secret sauce for turning defensive conversations into collaborative ones — find out how to unruffle feathers, bridge Engineering and Finance, and deflect the default reaction of “I can’t do that”.