Jeff McMillan, Founder and CEO of McMillan AI and former Head of AI at Morgan Stanley, joins Ray Rike to explain what separates AI programs that deliver measurable returns from those that stall in pilot mode.

Jeff led Morgan Stanley's partnership with OpenAI in 2022, when the firm became OpenAI's first financial services customer, and later ran the firmwide AI function focused on education, infrastructure, repeatable deployment processes and measurement.

The conversation covers the foundation companies need before they scale AI, how to build an accountable business case, the CFO's role in measuring ROI, and how governance can speed adoption instead of slowing it.

What Ray and Jeff Covered

• The AI stack is more than tools and models. Jeff argues most firms are underinvesting in four infrastructure layers: data quality across unstructured content such as PDFs, emails and Slack; a semantic layer that defines customers and products consistently; an AI governance layer that monitors other AI with humans above the loop; and an orchestration layer that will eventually coordinate hundreds or thousands of agents.

• Process redesign comes before AI investment. Many organizations cannot describe how work actually gets done, and senior leaders often do not know the steps in their own workflows. Putting AI on top of a bad process repeats automation's oldest problem.

• The business case starts with a benchmark. Teams should document current state throughput, cycle time, rework rate, and cost, then size the opportunity before building. A 4% efficiency gain is not worth pursuing, and production results need roughly three months of post-deployment data to validate.

• Efficiency only becomes ROI when you redeploy capacity. Creating 30% more capacity across 100 people produces no return if the time is not moved to higher-value work. Ray connects this to Parkinson's Law and to revenue per employee as a macro indicator of AI productivity.

• The CFO owns the measurement framework, not the outcome. Business owners define and capture the benefits. Finance builds the benchmarking framework, teaches teams how to measure and independently validates results, with an auditable record of labor, token and development cost for every project. Jeff recommends the KPI team sit in finance rather than operations.

• Risk-tiered governance and carved-out capacity. A risk-based tiering process matches approvals and monitoring to actual exposure, and a credible owner with both a risk and a commercial mindset keeps use cases moving. Jeff's top recommendation for CFOs and go-to-market leaders is to pick three high-impact, easier-to-deploy use cases and take the best person on the team out of their day job to deliver them.

If you are finding value in these conversations, please subscribe on your favorite podcast app, leave a five-star rating, and connect with Ray Rike on LinkedIn to suggest future guests.

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

This summer, three frontier labs found that their models had hacked real companies without anyone telling them to.

In this episode, Ray Rike and Peter Buchanan walk through the OpenAI, Anthropic, and Meta incidents, why all three traced back to the same third-party testing contractor and the same misconfigured sandbox, and what it means for every enterprise deploying AI agents. They cover the five questions buyers should be asking vendors now, the legal and insurance gaps that leave companies exposed, and why agent risk needs a named owner at the board level.

Key topics covered:

• How OpenAI models with weakened safety guardrails found a zero-day, reached the open internet, and broke into Hugging Face, leaving hundreds of thousands of coordinating notes for each other along the way

• Why Anthropic and Meta experienced nearly identical failures through the same third-party evaluator, and why "sloppy" understates a failure mode that repeated across three labs

• The enterprise readiness gap: 84% of organizations doubt they could pass an AI compliance audit of their agents, and only one in five keeps a real-time inventory of agent risk

• Five buyer questions, including whether vendors should provide safety certification the way SaaS vendors provide SOC 2 reports, and whether standard cyber insurance covers damage caused by your own AI agents

• Why the Computer Fraud and Abuse Act struggles to address AI-initiated attacks, and why contract language and insurance terms will move faster than new legislation

• Practical controls for labs and enterprises: default-deny outbound network access, immutable audit trails, an inventory of every privileged agent, human approval for high blast radius actions, and quarterly board-level agent risk metrics

Subscribe to the AI to ROI podcast, leave a five-star rating, and read the full August 18th edition at ai2roi.substack.com

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

When Dario Amodei published his essay calling on frontier labs to pace their releases, the leaders of the largest competing labs endorsed it within a weekend, and Microsoft followed on Monday.

In this episode, Ray Rike and Peter Buchanan examine what "pacing" actually means, why each lab defines it differently, and why self-governance by frontier labs leaves enterprises, governments, and emerging AI players without a seat at the table. They then make the case for formally designating frontier AI as part of U.S. critical infrastructure, with government-hired evaluators funded by lab fees.

Key topics covered:

• Anthropic's three-step pacing plan, including outside evaluators with employee-level access and a commitment of up to $1 billion over five years for Accenture to serve as an independent evaluator

• How the Anthropic (FAA model), Google DeepMind (FINRA model), and OpenAI (IAEA model) approaches differ, and why Cohere's Aidan Gomez compared the arrangement to the bond rating agency cartel that preceded the 2008 financial crisis

• CrowdStrike threat data showing 88% of observed exploits landed within 48 hours of disclosure, with AI agent activity generating 2.5x the event volume of human activity

• Why the government is under-resourced to respond: NIST's AI standards center runs on a $15 million budget against an estimated $84 million need, and CISA headcount has fallen from roughly 3,400 to about 2,300

• How the existing 16-sector critical infrastructure framework works, and a proposed tiered oversight model where only the most powerful frontier models carry the heaviest requirements

• What enterprise executives should do now: plan for exploit windows measured in hours, monitor agent activity separately from human activity, protect model weights, and build 72-hour incident reporting capability before regulation requires it
• 
  

Subscribe to the AI to ROI podcast, leave a five-star rating, and let us know which stories, topics, and guests you would like us to cover next.

Read the full September 22nd edition at ai2roi.substack.com

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.