Over 40% of agentic AI projects will be cancelled by end of 2027, according to Gartner. Many more remain stuck in experimentation. And even those that scale often fail to deliver the intended return on investment. The technology is not the problem. The mental models are.

In this first of two episodes based on a major new study by Johannes Habel and colleagues — validated through a survey of 167 technical experts — Kenza and Pascal unpack the four management illusions that undermine agentic AI at the use case development and deployment stages.

Illusion Domain 1 covers two illusions around developing use cases. The agent-first illusion is the belief that AI agents can and should eventually enter virtually any business process. In reality, only about 10% of automations in a large-scale transformation actually require LLM-based agents — the rest can be handled by simpler, more deterministic tools. The end-to-end illusion is the expectation that a single AI agent can reliably navigate complex multi-step processes. Today's agents require highly specialized scope and human orchestration; broader process improvement emerges by combining multiple agents with other technologies and humans in the loop.

Illusion Domain 2 covers the deployment stage. The testing illusion is managers' systematic underestimation of the effort required to evaluate and safeguard agent reliability. AI agents that perform well in demos regularly fail in unattended production — often silently — and the availability of domain experts, not technical experts, is usually the bottleneck. The scaling illusion is the assumption that a successful pilot can be replicated quickly and cost-effectively. In practice, processes that look similar often differ in inputs, IT systems, and data quality — and the human resource challenges of scaling AI agents are as significant as the technical ones.

Key Takeaways

• AI agents should be the last resort for automation, not the first. Only about 10% of identified automations in one large-scale transformation required LLM-based agents.
• The end-to-end illusion: today's AI agents cannot reliably navigate complex multi-step processes. The right approach is specialized agents for clearly defined subtasks, orchestrated alongside humans and other tools.
• AI agents that pass demos regularly fail in production — often silently. A voice agent in automotive called former customers to reactivate them; it performed significantly worse than human agents and caused many opt-outs.
• The testing burden falls on domain experts, not technical experts. Their availability — not the technology — is usually what limits deployment timelines.
• Scaling is not replication. A pilot designed to prove feasibility is rarely designed to scale. The second implementation almost always takes longer and costs more than expected.
• Scaling AI agents ultimately means scaling human agents too — reorganizing roles, building new skills, and securing employee buy-in.

Hosted on Acast. See acast.com/privacy for more information.

Show Notes

Most executive decisions are forecasts in disguise. 'Will this market grow?' 'Will this AI initiative deliver ROI?' 'Will this acquisition integrate well?' These are all probability estimates — and most leadership teams make them without any structured method, relying instead on experience, intuition, and the confidence of whoever speaks loudest in the room.

In this episode, Kenza and Pascal explore two landmark studies on AI-augmented forecasting and what they mean for executive decision-making. The first, by Halawi et al. (NeurIPS 2024), shows that a retrieval-augmented AI system can approach the forecasting accuracy of competitive human forecasters — and that combining AI and human judgment always outperforms either alone. The second, by Schoenegger et al. (2025, co-authored with Philip Tetlock, the world's leading expert on forecasting), shows that AI assistance improves human forecasting accuracy by 24–28% — even when the AI assistant is deliberately biased and overconfident.

The key insight: the benefit is not just informational. It is cognitive. Engaging with an AI perspective — even an imperfect one — forces assumption-surfacing, reduces overconfidence, and makes human judgment measurably more accurate. The Brier Score — a rigorous calibration metric that measures how well-calibrated probability estimates are, not just whether someone was right or wrong — confirms the improvement is real and quantifiable.

For boards making high-stakes decisions under uncertainty, this is not a hypothetical. It is a demonstrated, quantifiable capability that most executive teams are not yet using systematically.

Key Takeaways

• Combining AI forecasts with human judgment always produces better outcomes than either alone — confirmed across thousands of forecasting questions.
• AI assistance improves human forecasting accuracy by 24–28%, measured by the Brier Score — the gold standard for calibration.
• The benefit is cognitive, not just informational: AI forces you to surface assumptions you didn't know you were making and reduces overconfidence even in experienced forecasters.
• AI performs best exactly where boards struggle most — in the 30–70% uncertainty range where nobody is sure and overconfidence is highest.
• Most executive teams use AI to summarize and report. The studies suggest they should be using it to challenge assumptions before the decision is made.
• The practical starting point: pick one recurring decision type, introduce probability language, track outcomes, and score calibration over time.

Hosted on Acast. See acast.com/privacy for more information.

Show Notes

Nearly 9 in 10 organizations report using AI regularly. Only 37% say AI has made a positive contribution to their EBIT. And only 6% qualify as genuine AI high performers. The rest? Optimizing. Automating. Saving costs. Declaring victory.

In this episode Kenza and Pascal ask themselves: when does AI actually grow the top line?

And close with a board checklist and a final provocation:

"Do not ask what AI can do for your current business. Ask what business you can build because AI now exists."

Key Takeaways

• Efficiency asks 'how.' Growth asks 'what, for whom and why us?' These are different questions requiring different organizational conditions.
• Capability without mandate produces demos. Mandate without capability produces PowerPoint. Both without a business model hypothesis produce expensive activity.
• A Copilot license is not a strategy.
• The winners will not be the companies that bought the most AI. They will be the ones that used AI to become a different company while they still had the luxury of choice.

Hosted on Acast. See acast.com/privacy for more information.