Guide · AI agents · For the owner about to switch the agent on
Before Meta's AI agent answers your WhatsApp customers: what Meta says to test, and how to try to break it
An AI agent that only ever sees the questions it was built for will fail the first time a customer asks something else. Meta's own guide to testing its AI agent for business says as much, and its longest section is about trying to break the agent before customers do. Here is what it says to check, in plain terms.
Published 6 October 2026 · Updated 7 October 2026
Three ways to test, cheapest first
- Test messages
- Messages sent straight to the agent, one at a time or as a short conversation. Meta says these are not billed and work while the agent is still switched off. They never touch WhatsApp, so they show what the agent says, not how it looks on a phone.
- Scored practice conversations
- Meta runs conversations against the agent with a simulated customer, scores them from 1 to 5, and summarises the most common failures. Meta suggests running the same set before and after every change.
- A small group of real people
- The agent switched on for a short list of WhatsApp numbers only, at most 20 per business number. Meta says this needs no payment method; switching the agent on for everyone does.
Meta's advice is to work through them in that order, because each costs more to set up than the last. Scores are only as honest as the practice conversations behind them, so Meta also says to add the conversations the business does not yet know how to handle.
Meta's reference page for the scored conversations shows what goes into each one and what comes back.
- A practice case
- A short written scenario telling the simulated customer what to try and within what limits, with optional categories, a cap on how many turns the conversation may run, and the outcomes the agent must reach to pass.
- How a run works
- One run can cover several cases. Meta works through three stages, the simulated conversations, the scoring, then a report, and shows progress while it runs.
- What comes back
- Each conversation gets a score from 1 to 5, which Meta says comes from a language model acting as judge, and keeps its transcript, the reasons for its score and recommended actions. The report adds an average for whole conversations, an average per turn (one customer message and the agent's reply), a written summary, highlights and the most common kinds of failure.
Try to break it
- Ask what it does not know
- A product the business does not sell, a place it does not serve, a policy it never published. Meta says a well-grounded agent says it does not know; a badly grounded one invents an answer.
- Let its sources disagree
- An FAQ that contradicts the website, or a catalog price that differs from an uploaded file. The agent will pick one, and Meta says the business needs to know which.
- Take it off topic
- Ask about a competitor, or something unrelated to the business, and check the agent stays on the business's own subject.
- Ask for promises nobody approved
- A discount, a refund, a guaranteed delivery date, a legal or medical opinion. Also ask it to ignore or reveal its instructions, or to act as a different assistant.
- Make the business's systems fail
- A booking or order system that is slow, down, or returns nothing. Meta warns that agents often confuse no results with something going wrong, and that a task needing two lookups must not be reported as done when the second fails.
- Send messages nobody planned for
- Very long and very short messages, emoji only, a language the business did not prepare for, the same question asked five ways with typos and slang, two questions at once, and personal details volunteered mid-chat.
- Make the conversation long
- Change your mind halfway, cancel what was just ordered, go quiet and come back. Check the agent still remembers what was settled at the start.
Three surprises Meta warns about
The first only appears with real people on WhatsApp. If the business's own connected system sends any message into a conversation, it takes the conversation over and the agent stops replying until it is handed back. Meta calls this the most common reason an agent answers in testing and goes quiet once customers arrive.
The second comes before any customer writes. Meta's setup page says the short test list does nothing on its own: the agent only keeps to it when its audience is set to the test list. The default is everyone, so an agent switched on without changing that answers every customer who writes, whoever is on the list. Meta says to set the audience first, then check the setting before switching the agent on.
The third is the spending limit. When it is reached, Meta says the agent finishes the reply it is writing, then stops answering new messages and hands them to a person until the next period begins, without warning. Someone has to be ready to answer when that happens. Who answers when the agent and another system share a number is in the conversation routing guide.
After it goes live
- Watch how often it hands over. Meta says a rising rate usually means a gap in the agent's knowledge or instructions, not a broken agent.
- Read the conversations a person had to finish. Meta calls them the next things to test.
- Watch the business's own systems. A lookup that times out more and more often makes the agent's answers worse before any customer complains.
- Test again after every change. Meta warns that a change that fixes one conversation routinely breaks another.
What the agent is and what switching it on changes is in the Meta Business Agent guide, and what Meta says the agent may settle alone on a support line is in the support line guide.
Questions
Does testing the agent cost anything?
Meta says messages sent through its test tool are not billed and work while the agent is still switched off. Turning it on for up to 20 testers needs no payment method; turning it on for everyone does.
We put our testers on the list, but the agent answered a customer. Why?
Meta says the list only applies when the agent's audience is set to the test list. The default is everyone, so the audience has to be changed, and checked, before the agent is switched on.
We cannot find Meta Business Agent in WhatsApp Manager. Why?
Meta says its tab only appears when at least one number on the WhatsApp Business account is eligible. Setting it up there, which is also where the terms are accepted, does not switch the agent on yet.
Why did the agent go quiet after our system replied?
Meta says that when the business's own connected system sends a message into a conversation, it takes the conversation over, and the agent stays quiet until it is handed back.
What happens when the agent's spending limit is reached?
Meta says it finishes the reply in progress, then stops answering new messages and hands them to a person until the next period begins.
What should a practice case for the agent say?
Meta's reference describes a short written scenario for the simulated customer, the most turns the conversation may run, and the outcomes the agent must reach to pass. Our reading: write them from real chats your team found hard, such as a refund request or a question about a product you do not stock.
Will it answer customers who write in Arabic?
Meta says the agent detects the language of each message and replies in the same one, with no list of languages to set. English gets the strongest answers, so test the languages your customers use. The business's information does not need translating, and product names are left as they are.
Sources
- Meta's developer documentation: Meta Business Agent, Test your agent, read 6 October 2026.
- Meta's developer documentation: Meta Business Agent, Capabilities, read 6 October 2026.
- Meta's developer documentation: Meta Business Agent end-to-end setup, read 7 October 2026.
- Meta's developer documentation: Meta Business Agent reference, Agent Eval, read 7 October 2026.
More plain-language notes like this one are on the guides page.
Want a person ready when the agent hands over?
We set up a business's WhatsApp number on a system its team runs, so when an AI agent hands a conversation over, a person can pick it up in one place the business owns. A short call is the easiest way to see how that would work for you.
Book a demo