AI agents and scheduled automations are not set it and forget it. The requirements you start with are an educated guess, and you often discover what you need only after the agent does something you don’t need. In this episode of The Missing Manual, Julia Levy and Jenny Cotie Kangas explain how to tune an agent once it is running, and why telling the AI where it falls short is the step most people skip.
Jenny starts with a question from a recent workshop. An attendee built a Gmail agent to triage her inbox, and the agent kept stopping to ask for permission. The fix was in the architecture. An agent connected to Gmail on the back end should not need permission every time, but one working through the browser will. Jenny then describes her own signal brief, which drafted replies in her voice across email, WhatsApp, LinkedIn, and text. If she left one email unanswered for five days, she came back to five drafts of the same reply. She rebuilt the agent so it shows her where her attention needs to go, and then they work through each response together.
Julia adds that AI models change, so anything built on a model can drift over time. She recommends putting regular reviews of your automations on the calendar, the same way you would review a website. She has Claude interview her at the start of a build and ask what she isn’t thinking of. Then she runs the same conversation through Gemini and ChatGPT to catch what she missed. Jenny suggests going one step further with a scheduled automation that reviews your other automations, checking which model each one uses, its outputs, and its token use.
The closing advice from both hosts is direct. If an agent frustrates you, say so to the AI, because it cannot match your idea of good until you define what good is and what it isn’t. Then have the AI ask you questions and challenge you rather than agree with everything. The Missing Manual is also opening up to guests, so if you are building something interesting with AI, Julia and Jenny want to hear from you.
Find The Missing Manual wherever you listen to podcasts.