Cover of Agents Are Overtaking Humans in Search: Why Search Must Be Rebuilt

Agents Are Overtaking Humans in Search: Why Search Must Be Rebuilt

Agents Are Overtaking Humans in Search: Why Search Must Be Rebuilt

Listen for free

View show details

Season's Savings | 0.99 €/month for 3 months

5.99 €/month thereafter - terms apply. Cancel monthly.
Summary
 Saahil Jain joined You.com as a founding engineer in 2020, when it was a consumer search engine competing with Google. Now its CTO, he builds search for AI agents, and he argues that however smart models get, they will still need tools to find out what is happening in the world. His analogy: we built the self-driving cars before the roads. He walks through which work he expects to move inside the model, why better models made harnesses matter less, and why cheaper open models and routing are giving harnesses a new job. He says he changed his mind on that last point in the past few months. Then we get into how agents search differently than people do: thousands of queries at once, recall over ranking, and a much wider range of latency needs. Saahil traces a query from fan-out to the index to re-ranking, and explains why You.com judges search by whether the agent answered the question, and why he trusts customer evals and benchmarks that change over time more than static public ones. We cover: - Core intelligence versus augmented intelligence, and the self-driving cars that arrived before the roads- Which tasks Saahil expects models to absorb, and where unverifiable rewards slow them down- Why frontier labs keep going vertical while open-model labs give weights away- How stronger models shrank the gap between good and bad harnesses, and how routing across open models can revive them- How You.com post-trains models with its search tool while trying not to overfit to any one model- How agent search differs from human search: query volume, context, ranking and latency- The life of a query: subqueries, lexical and semantic indexes, re-ranking and enrichment- Why static benchmarks can be memorized, and what dynamic benchmarks and customer evals measure instead Mentioned in this episode:- I Paid an AI Agent $8 to Write About its ‘Life’: https://newsletter.chainofthought.show/p/i-paid-an-ai-agent-8-to-write-about- MotionBench: https://motionbench.sh Connect with Saahil Jain: - LinkedIn: https://www.linkedin.com/in/saahiljain/- Twitter/X: https://x.com/saahil9jain- You.com: https://you.com- You.com API docs: https://you.com/docs/quickstart Connect with Chain of Thought host Conor Bronsdon:Newsletter: https://newsletter.chainofthought.show/Twitter/X: https://x.com/ConorBronsdonLinkedIn: https://www.linkedin.com/in/conorbronsdon/YouTube: https://www.youtube.com/@ConorBronsdon More episodes: https://chainofthought.show Chapters:(0:00) Agents run a thousand searches at once(0:25) What survives as AI models get smarter(1:47) Is this the singularity?(4:26) Core intelligence versus augmented intelligence(6:33) Self-driving cars before the roads(7:27) What belongs inside the model(11:00) Where verifiable rewards run out(12:16) Can agents form relationships?(16:58) What model labs own, and what independents can build(20:21) What a harness is(22:39) The model is eating the harness(23:14) Open models and routing revive the harness(25:32) Open versus closed: what the labs keep(31:30) OpenAI, Baseten and the enterprise model budget(33:53) Co-evolving a search tool with the model labs(37:41) How agent search differs from human search(42:42) Life of a query, from fan-out to index(47:22) Measuring retrieval by whether the agent answered(50:21) Static benchmarks, freshness and customer evals(54:19) Why he changed his mind on harnesses Thanks to Walrus, presenting sponsor of season four of Chain of Thought. Walrus Memory gives AI agents portable, verifiable memory that carries context across apps, sessions, and other agents. Get started at https://walrus.xyz/cot Thanks to CoralBricks for sponsoring this episode. CoralBricks serves open models through one API, fast and at low cost: DeepSeek V4.1 Flash streams at over 300 tokens a second. Get $5 in credits on signup, plus another $5 on your first purchase: https://www.coralbricks.ai/r/Z6V2RE Thanks to Inngest, presenting sponsor of season four of Chain of Thought. Agents in production run long: they call models and wait on APIs and people. But the longer agents run, the more they break. Inngest handles that with durable execution. You build your agent as steps in TypeScript, Python, or Go. When a step fails, Inngest retries it with exponential backoff, and completed steps are saved and skipped. Try it out: https://inngest.link/cot-pod Thanks to WisdomAI for sponsoring this episode. In WisdomAI's latest benchmark, Claude Code alone topped out at 73% accuracy; WisdomAI closed the gap, delivering accurate answers 3x faster at about a third of the cost. Get "The Blueprint for Trusted AI Analytics": https://wisdom.ai/benchmark
adbl_web_anon_alc_button_suppression_t1
No reviews yet