Three Lessons from Building AI Agents

Building an agent is easy. Building an agent that improves your work is hard.

A first version comes together fast, and it looks impressive. The hard part comes after: making sure it’s right, making it reliable, and knowing what it should never decide on its own.

Agents don’t just make work faster. They make mistakes faster too, and with a confident tone that makes them hard to spot. So most of what I’ve learned has less to do with the AI and more to do with the design decisions around it: how to split the work, how to check it, and where a person stays in the loop.

Here’s what I’d tell anyone building their first agent.

1) Never trust the first pass

An agent can find a source, a number, or a name and be convincingly wrong about it: the right-sounding answer from the wrong document, a person with the same name, a figure that was true two years ago. And it will say so with total confidence. A wrong detail is often worse than a missing one, because nobody questions it until it causes a problem.

So every stage that generates something is followed by a stage whose only job is to check it. 

Research gets verified against other sources. Scores get a second look before anything moves forward. Verification isn’t polish you add at the end. It’s what makes the output safe to use.

2) One agent, one job

It’s tempting to build a single agent that researches, evaluates, decides, and writes up the result. In my experience, splitting the work just works better. When I designed account research, for example, I used two agents: one looks for priorities a company has shared publicly, and the other looks for changes in its leadership.

They draw on different sources, produce different signals, and fail in different ways. Narrow agents are easier to debug, easier to verify, and easier to reuse.

3) The agent finds the signal; a person makes the call

Agents are great at surfacing information. They’re less reliable at judging what it means or what to do about it. When I review agent output, a lot of it is factually right but off the point: a true fact attached to the wrong question.

A person reading the output catches that in seconds. So we treat agent output as a well-researched first draft, not a final answer. Judgment is the part that should stay human.

Agents make work faster. Whether they make it better depends on everything around them: checks that catch mistakes, narrow jobs that keep each step honest, and a person who decides what to do with the result.

Get those right, and you can move fast and still stand behind the result.

About the Author

Related Personalization Practices

What Deel’s “Teams” Pages Get Right About Personalizing for the Buying Committee

B2B personalization needs to go beyond an industry or company value prop to speak individually to each member of the buying committee A purchase is

Read More

Personalization and AEO Start with the Same Thing: Intent

Personalization and Answer Engine Optimization (AEO) are often treated as separate initiatives. One is about creating a website experience that feels relevant to the visitor. The other is about making content easier for AI-powered search engines to discover and cite.

Read More

The Best Websites Don’t Just Educate. They Convert.

One of the most common recommendations in B2B marketing today is to invest in thought leadership. It's good advice. Buyers want expertise. They want insights. They want help understanding complex challenges before they engage with a vendor.

Read More