AI is moving quickly, but getting from an impressive demo to something that actually works in production is a very different challenge.

In this episode of The Databricks Diaries, Dan Thornton sits down with Chase E., Head of AI FDE, EMEA at Databricks, to talk about what it really takes to turn AI ideas into production-ready solutions.

Chase shares his perspective on why so many AI POCs fail, the importance of strong data foundations and governance, choosing the right AI use cases, and why businesses need to think beyond simply cutting costs.

They also discuss the role of talent and internal capability, how organisations can avoid AI sprawl, and why building a centralised AI capability or community of practice is becoming increasingly important.

The conversation covers:

• Why a demo isn't the same as production

• Why AI POCs and MVPs fail

• Why AI can't fix weak data foundations

• Governance and the importance of getting it right early

• Choosing AI use cases based on real business problems

• Moving beyond cost reduction and towards revenue generation

• Building internal AI capability through co-delivery

• The risk of AI sprawl across organisations

• Why a centralised AI capability can help organisations scale

A great conversation for anyone working across data, AI, engineering, technology or digital transformation.

Guest: Gavin Goodland, Chief Data Officer at National Grid

AI is creating extraordinary opportunities for businesses, but it is also creating an enormous new demand for energy.

In this episode of Databricks Diaries, Andy Davis sits down with Gavin Goodland, Chief Data Officer at National Grid, to explore the relationship between AI, data centres and the infrastructure required to power an increasingly digital economy.

As hyperscale data centres add significant new loads to the electricity network, National Grid is simultaneously using AI, real-time data and advanced optimisation to help manage a grid that is becoming more complex, distributed and dynamic.

Gavin explains how AI is being applied across National Grid — from predicting equipment performance and optimising network capacity to physical AI, drones, workforce training and intelligently managing demand from data centres.

The conversation also goes beyond individual use cases into a bigger question facing every enterprise: how do you turn rapid AI innovation into sustainable business value?

They discuss:

• Why AI and data-centre growth are becoming a major consideration for the electricity grid
• How AI can help balance demand, generation and network capacity
• Using real-time operational data to get more value from existing infrastructure
• Physical AI, robotics and drones across critical infrastructure
• Why National Grid starts with the business problem rather than the technology
• The importance of keeping the “human in the lead”
• Moving employees from curious, to comfortable, to confident with AI
• Where organisations should build AI themselves versus buy from technology providers
• Why legacy technology shouldn’t prevent organisations from innovating
• The emerging importance of an AI control plane for integration, observability, lineage and governance
• Managing the rapidly changing economics of AI and consumption-based technology
• Why Gavin believes organisations should establish an AI Value Office to evaluate value and risk consistently

A fascinating look at AI from an organisation sitting at the intersection of two of the biggest technology and infrastructure shifts happening today: the growth of AI and the transformation of the energy system.

Is AI making technical skills less important?

Or is it actually making experience, judgement and technical understanding more valuable?

In this episode of Databricks Diaries, Dan Thornton is joined by Sham and his son Milo for a conversation about AI, data engineering, careers and what companies should really be looking for when hiring data talent.

They get into:

• What being “AI-ready” actually means

• Why strong data foundations matter before AI can give you useful answers

• How AI helped turn work that could have taken 2–3 months into a couple of weeks

• Why you still need to understand and review AI-generated work

• How AI is changing the day-to-day role of data professionals

• What Milo looks for when hiring data analysts

• Why curiosity and problem-solving matter when assessing candidates

• What makes a strong modern data engineer

• Whether companies should prioritise experienced AI-savvy engineers over junior “vibe coders”

There’s also a more personal side to the conversation, with Sham and Milo talking about their different routes into technology, career advice and what they’ve learnt from each other.

The big question:

Has AI actually made technical experience less valuable — or more valuable?

What do you think?

Connect with Dan Thornton and Primus Connect for specialist Databricks recruitment.

#Databricks #AI #DataEngineering #DataAnalytics