Four things I would check before connecting an AI agent to your data

Why an AI agent makes up balances and how to prevent it: history with results, lookups in a loop, low temperature and keys kept out of the app.

JZ Jorge Zamarrón 07 Nov 2026 3 min read
Four things I would check before connecting an AI agent to your data

Recaudo, our WhatsApp sales and collections product, has sent more than 2.3 million messages and handled close to 495 thousand conversations for 224 businesses in ten countries, and it has AI agents that answer on WhatsApp. Building agents for Recaudo and for clients, we got things wrong several times, and almost every mistake was a design mistake.

This is for anyone about to put an AI agent to work on company data: an ERP, a CRM or a customer database. These are the four things I would check first.

1. The agent has to see what it looked up

In an agent we built for a CRM, the model made up balances and due dates. The conversation history stored the user’s questions and the agent’s replies, but not the results of the database lookups. On the next turn the agent remembered it had said a number, could no longer see where the number came from, and filled the gap with something plausible.

The rule we kept: every lookup and its result go into the history, just like the messages. If the agent cannot see the data, it cannot use it.

2. One lookup is rarely enough

A question like “how much do I owe and when is it due?” looks like one lookup. In practice it is three: find the customer, fetch their contracts and compute the balance of each one. An agent that can make only one lookup per turn answers with the first thing it found.

So the agent has to work in a loop: look something up, read the result, decide whether it needs another lookup and only then answer. The loop needs a limit, and when the agent reaches it without an answer, the right move is to hand the conversation to a person.

3. For data, creativity gets in the way

Models have a setting, temperature, that controls how much their answers vary. For a welcome message a high temperature gives more natural sentences. For a balance it gives different numbers to the same question. An agent that touches money, contracts or inventory should run at a low temperature and get its friendly tone from the instructions.

4. Keys do not ship inside the app

It sounds obvious, and we have found it in production systems: the AI provider’s key stored inside the mobile app or the browser code. Anyone who downloads the app can pull it out and use your account. The call to the model has to go out from the server, where you can also check what data is sent and how much is spent.

What we connect

We have connected systems to SAP Business One, NetSuite, in-house databases and the WhatsApp Business API, and for agents we increasingly use MCP servers, an open standard that lets a model use a system’s tools with defined permissions. The groundwork stays the same: deciding what the agent can read, what it can change and when it hands the conversation to a person.

That groundwork looks more like process consulting than programming. If the collections process has no clear rules for a person, it will not have them for an agent either.

Do you have an agent in production that sometimes answers with things that are not in your data? I would like to know which of these four points is the problem, or whether it is one I do not have on the list.

JZ

Written by

Jorge Zamarrón · Founder & CEO

Jorge Zamarrón founded Zygnus and has fifteen years of software and process consulting for companies. Jorge sits on the board of Alcis Corp, a holding of more than ten companies with combined sales above 12 billion pesos a year, and leads Innovation and Development at J. García López.

Newsletter

One email when we publish something new

No promotions. Just the articles.