# Tests (/docs/agents/voice-agents/tests) > For the complete documentation index, see [llms.txt](https://docs.tandemcx.ai/llms.txt). Any page on this site can be fetched as Markdown by appending `.md` to its URL. A test is a conversation the platform holds with your agent and then grades. Run your tests against a draft before you publish it, and you find out whether a prompt or tool change broke something that used to work. Tests run over chat. Chat builds the same agent a phone call does (prompt, model, tools, and knowledge), without speech recognition and text-to-speech. A test with a **Caller Number** runs with the call wording of the [platform skills](/docs/agents/voice-agents/skills#platform-skills). A test without one runs as a website chat, where `collect-code` and `verify-identity` use chat wording and `read-back-values` is skipped. Tools that only exist on a call run against stand-ins the test sets up: a caller number and contact, keypad entries, workflow variables, and a caller-memory record (see [Call inputs](#call-inputs)). A test never reads or writes your real contacts, what the agent remembers about a real caller, or a dropped call, and never switches to another agent. Settings that act on the audio of a call aren't covered: **When the caller goes quiet**, **Thinking cue**, **Allow barge-in**, **Voice activity detection**, **Turn detection**, and **Maximum call duration**. Call recording tools can't be tested either. ## Open your tests Open an agent and select **Tests** in the editor header. The **Review** step shows the status of the version on screen as a badge: - **Passing 12/12** — 12 of the 12 tests passed on this version as it is now. When the first number is lower, the rest haven't run on this version yet or couldn't run on it. - **3 failing** — three tests failed or ended in an error on their latest run. - **Running 4/12** — a run is going, and 4 of its 12 attempts have finished. - **Not run** — the tests have never run on this version. - **Stale** — they ran, but the version changed since. An agent with no tests shows no badge. The Tests page opens on the version you came from. Pick another in the **Version** list to see its results; the published version is marked **(published)**. Tests belong to the agent and aren't versioned, so every version runs the same tests. The **Agents** list has a **Tests** column with each published agent's result on its published version, using the same badges as above; select a result to open that agent's Tests page. A published agent with no tests shows **No tests**, which opens its Tests page too, and an agent that isn't published shows a dash. The column is hidden first in a narrow window. ## Write a test Select **Add Test**. Give the test a **Name** and, optionally, a **Description**, then pick a **Caller**: - **Scripted** — sends the **Caller Messages** you write, one at a time, each after the agent replies. Use it when you know exactly what a caller says. - **Simulated** — has an AI play the caller. You give it a **Persona** (who the caller is and what they know) and a **Goal** (what they want). It uses only the facts in the persona, and it talks to your agent until it decides the goal is reached or can't be reached, or until **Max Turns** runs out. Use it to try the conversations you didn't think to script. To change a test, select **Edit** on its row. **Duplicate** and **Delete** are in the row's menu. At 50 tests, **Add Test** and **Duplicate** are unavailable. ### Tool responses Tools don't run for real in a test unless you ask. Each tool answers with an empty result, or with the JSON you enter as its **Response** under **Tool Responses**. A test can't send a text message or book an appointment by accident that way, and a test of your booking flow doesn't depend on what's on the calendar today. Turn on **Call for real** for a tool to run the actual tool. The editor warns you that the tool can change data outside the test unless the platform knows it's read-only: an integration tool that sends a `GET`, a Zendesk tool set to **Show Ticket**, or `get_current_time`. A test with a tool on **Call for real** isn't run again after an interruption, because the tool may already have run; it ends in an error instead. `capture_dtmf`, `set_workflow_variable`, and `transfer_to_voice_agent` answer from the test's [call inputs](#call-inputs) unless you give them a response, and they aren't marked **Stubbed**. `update_contact` needs a **Contact**, and `lookup_contact` a **Caller Number**. With **Call for real** they run against the test's contact, never a real one, and `lookup_contact` finds only that contact. A tool marked **Call for real** that reads the caller's number, such as an integration tool that uses `{{sys.telephony.ani}}`, receives the test's **Caller Number**, so it can reach the real system for that number. Some tools can't take a response because they run inside the agent. `end_call`, `search_knowledge_base`, `skills`, and [agent tools](/docs/agents/voice-agents/tools#add-an-agent-tool) always run for real. The linked agent behind an agent tool runs too, and its own tools answer from your tool responses like any other tool. A [workflow handoff](/docs/agents/voice-agents/tools#the-six-kinds-of-tool) can't take a response either. It ends the test without running the workflow, and a successful `transfer_to_external` or `transfer_to_voice_agent` ends the test the same way, as **Handed off**. The other agent doesn't run, so give it tests of its own. A transfer the agent is refused, such as to an agent with no published version, doesn't end the test. The **Tool** picker lists the version's tools. To name a tool the list doesn't show, type its name. That is how you reach a tool of a consulted agent, named after the agent tool, two underscores, and the tool (`billing__lookup_invoice`), or a tool from an MCP server. Name a tool the way the version's tool list shows it. Saving refuses a name that isn't a tool, such as a setting's name (`transferToVoiceAgent` for **Transfer to another agent**) or, when you save against a version, a spelling of one of its tools with different capitals or underscores (`FindSlots` for `find_slots`), and says which tool you meant. Saving also checks tool and disposition names against the version you have open, so a test written for another version may be refused with "is not configured on this version". ### Call inputs Under **Caller Messages**, **Add Keypad Entry** adds keys the caller presses: `0` to `9`, `*`, and `#`, up to 32. Keypad entries right after a message are what the caller types when the agent's reply calls `capture_dtmf`, in order. A `capture_dtmf` with no entry left gets nothing, as when a caller presses no keys. An entry no capture took is sent as its own turn, the way a call delivers keypresses with **Keypad Input** on: only its first group, ending at the agent's end key (which isn't sent) or its maximum digits. Without **Keypad Input** it's ignored. A keypad entry that follows another keypad entry is a turn of its own, so it can't answer a `capture_dtmf` that the earlier one triggered. A **Response** you set for `capture_dtmf` stubs the tool, so it takes no entries. Every queued entry is then sent as a keypad turn after the message (or ignored, without **Keypad Input**). In a test the `capture_dtmf` prompt is only a tool argument and isn't spoken, so a **Reply contains** check on that prompt doesn't see it, and a criterion sees it only as the tool call's argument. `set_workflow_variable` is offered in a test whenever it is enabled, but on a call only when the agent resumes a workflow at the end. These sections of the editor apply to scripted and simulated tests alike: - **Caller Number** — the caller's **Phone Number**, in E.164 (`+15551234567`). It fills `{{sys.telephony.ani}}`, and the contact and memory belong to it. Leave it blank for a caller with no number. - **Contact** — the **Contact** switch makes the agent find a contact for the caller number, with the **Attributes** you add (each a **Name** and a **Value**). It needs a **Caller Number**, and contacts turned on for the agent. The contact exists only in the test and nothing is saved to it. On a call to a contacts-on agent, every caller with a number has a contact. To test a first-time caller, turn **Contact** on and add no attributes. - **Caller Memory** — the **Caller Memory** switch makes the caller a returning one, with **Prior Calls** and up to three notes: **Synopsis** (read at the Warm memory depth), **Summary of Prior Calls** (Light and Continuity), and **Open Items** (Continuity). It needs a **Caller Number** and a [memory](/docs/agents/voice-agents/brain#memory) depth other than **None**. Only a custom returning-caller greeting applies, as on a call. A rewritten greeting doesn't, because the test reads the base greeting. With the switch off, memory is off during the test. - **Workflow Variables** — values the calling workflow passed in, read as `{{vars.name}}`, each a **Name** and a **Value**. `set_workflow_variable` can set only a variable the version declares, and the **Sets a workflow variable** check picks from those. Nothing real is touched. The contact and memory record are built for each turn from the test; no real contact is read or changed, no caller-memory record is read or written, and no dropped call is filed. Saving a test is refused when an input can't be used: - `Tool "update_contact" needs a caller contact. Turn on Contact.` - `Tool "lookup_contact" needs a caller number. Set one under Caller Number.` - `Contacts are off on this version, so the test's contact is never read.` - `Caller memory is off on this version, so the test's memory record is never read.` - `Keypad entries need Keypad Input or keypad capture on this version.` - `Workflow variable "" is not declared on this version.` ### Checks and criteria A test passes only when every check and every criterion passes. **Checks** are rules the platform evaluates in code. Add one with **Add Check** and pick its **Type**: | Type | Passes when | | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Calls a tool | The agent called the tool. Add **Arguments** as a JSON object to require those values; arguments you leave out can be anything. Set **Times** to require exactly that many calls. | | Never calls a tool | The agent never called the tool. | | Ends with a disposition | The conversation ended with that disposition. | | Agent ends the conversation | The agent did, or did not, end the conversation itself. | | A reply contains | Any agent reply contains the text (ignoring case) or matches the regular expression. Turn on **No reply may match** to require the opposite. The greeting counts as a reply. | | Finishes within N turns | The agent took N turns or fewer. The greeting doesn't count as a turn. | | Sets a workflow variable | The agent recorded the variable with `set_workflow_variable`, with exactly the **Value** if you give one. | A regular expression with a quantified group inside another quantifier, such as `(a+)+`, is refused when you save. A pattern that still takes too long to evaluate fails its check. **Criteria** are plain-language expectations, such as "Confirms the new appointment time before ending". Add one with **Add Criterion**. An AI judge grades each one against the transcript and gives a reason. A criterion the transcript can't answer fails with the reason "no evidence". Prefer a check when one can express what you expect. A check gives the same answer every time for the same conversation. ### Repeat An agent doesn't answer the same way every time. Set **Repeat** to run a test up to 5 times. It passes only when every attempt passes, and its row says how many attempts passed, such as "2 of 3 attempts passed". Leave **Repeat** blank to use **Default Repeat** from the test settings. ## Recommend tests Select **Recommend Tests**, beside **Add Test** and in the empty state, to ask for up to 3 suggested tests for the version you have selected. They're written from that version's setup (prompt, tools, transfers, and dispositions) and up to 20 of the agent's most recent calls. An agent with no calls yet gets tests from its setup alone. The button reads **Writing tests…** while it works, usually under a minute. When it finishes, a toast says how many tests it added, "No new tests to suggest", or "Couldn't write tests. Try again." If 10 suggestions are waiting for the agent by the time it runs (coaching, for example, can add one while you wait), it says "This agent already has 10 suggestions waiting. Accept or dismiss some first." instead. The tests land under **Suggested Tests** below, on the same page. **Recommend Tests** is unavailable while 10 suggestions are already waiting for the agent; accept or dismiss some first. ## Make a test from a conversation Open a finished call or chat in **Insights → Call Logs** and select **Make a Test**. The button shows only for a conversation a voice agent handled. It opens that agent's Tests page with a new scripted test in the editor, named after the conversation: - **Caller Messages** — what the caller said, in order. - **Tool Responses** — each tool the agent called, answering with the last result it returned that wasn't an error. A tool that only returned errors, or a tool that can't take a response, gets none. - **Checks** — a **Calls a tool** check for each tool the agent called, and an **Ends with a disposition** check for the disposition the conversation ended with. A draft can include `capture_dtmf`, `set_workflow_variable`, and `transfer_to_voice_agent` calls. It leaves out `update_contact` and `lookup_contact`, which need a **Caller Number** and **Contact** you add yourself. The test isn't saved until you select **Save**, so trim it first: drop the caller messages and tool responses the test doesn't need, and add the criteria a good answer meets. A draft holds up to 40 caller messages and 20 checks, and stays small enough to save. When the conversation has more, the draft keeps the earlier messages and tells you so. A website chat keeps only its last 60 messages, so a draft from a long chat starts partway through it. ## Suggested tests Suggestions appear under **Suggested Tests**, at the top of the agent's Tests page, and on the agent's page in **Automations → Agents → Coaching**. One list, two sources: **Recommend Tests** above, and [coaching](/docs/agents/coaching#agent-changes), which can also suggest tests built from the calls it reviews, up to 2 per review. Unless the agent has [automatic reviews](/docs/agents/coaching#automatic-reviews) on, a coaching review reads only the calls a supervisor coached or took over, so that's where its suggestions come from. After you apply a coaching recommendation, it suggests one more test that checks the change holds; that card is labeled **Guards an applied change**. On the Tests page, a suggestion from coaching is labeled **From Coaching**, so you can tell the two sources apart. Each card gives the reason for the test and, when one is cited, a quote from the call it came from. **Accept** opens the test in the editor: in place on the Tests page, or on the agent's Tests page when you accept from Coaching. Saving it creates the test and marks the suggestion accepted; leaving without saving keeps it waiting. **Accept All**, on the Tests page, saves every waiting suggestion as a test. A suggestion the server can't save (for example, because it would pass 50 tests) stays waiting with its error shown on the card, and the rest are saved. A toast reports how many saved, such as "Saved 2 of 3". Suggested tests never turn on **Call for real**, and **Accept All** leaves waiting any older suggestion that does, so you review it in the editor first. **Dismiss** clears a suggestion, and the toast that follows carries an **Undo**. No suggestions are added while 10 are waiting for an agent. Suggestions come from voice calls only, because coaching reviews calls only, and they're kept for 90 days. ## Write tests with Tandy In the Tandy chat, type `/writing-agent-tests` and describe the test you want, or ask for a test from the call or chat you have open. Tandy reads the agent and its existing tests, writes the test, and shows it on a card naming the agent, the test, the caller, and how many checks and criteria it has. **Create** saves it. Tandy can also run the agent's tests and read you the results. A run uses model tokens, so Tandy asks for your approval first. It works through the tools listed under [the Tandem MCP server](/docs/overview/building-with-ai#the-tandem-mcp-server), so an assistant you connect yourself can do the same. In the test list, a test Tandy wrote shows a **Tandy** badge, and one saved from a suggestion shows **Suggested**. ## Run tests Select **Run All** to run every test on the version, or the play button on one test's row. One run per agent goes at a time. While it runs, the page shows how many attempts have finished, such as "1 of 5 attempts finished", and **Cancel Run** stops it. **Run All** and each row's play button do nothing until it ends, and say "A run is in progress" when you point at or tab to them. When a run finishes while you're on the page, a message summarizes it in one of three ways: "All 5 tests passed" (or "1 test passed"); "2 of 5 tests failed", which counts **Failed** and **Error** tests (an error counts as failed here, as it does in the editor badge); or, when no test failed but some didn't pass, a warning such as "4 of 5 tests passed". A test that is **Invalid**, didn't finish, or was edited or deleted during the run doesn't count as passed. **View Results** in the message lists every test in the run, and the **×** closes it. A cancelled run shows no message, and neither does a run that finished before you opened the page. Each test's row shows its result from the newest run on this version that included it, so running one test keeps the others' results. A run that ended before it reached a test, because it was cancelled or couldn't start, leaves that test's earlier result in place. Editing a test clears its result, and the row reads **Not run** until the test runs again. A result is one of: - **Passed** or **Failed** — every check and criterion passed, or at least one didn't. - **Error** — the test couldn't finish, for a reason that isn't your agent failing it: a model didn't respond, the test ran past its time limit, or the version was edited during the run. The result says which. An error counts as failing in the editor badge. - **Invalid** — the test names a tool or disposition this version doesn't have, uses a tool without the call input it needs, or sets a call input the version doesn't read (a contact with contacts off, memory with memory off, keypad entries with neither **Keypad Input** nor keypad capture, a **Sets a workflow variable** check on a variable the version doesn't declare). It didn't run. This includes a **Tool not called** check on a tool the version doesn't have, which can never fail. Each row's badge shows the result. Under the test's name, a test that repeats says how many attempts passed, such as "1 of 3 attempts passed" (or, while it runs, how many finished), then how long ago the newest attempt finished, such as "5m ago". To open the result, select the row, or select **View Results** in the row's **⋮** menu. The result shows each check and criterion with its outcome and reason, then the transcript. The row you opened is highlighted. A criterion is marked **AI judged**, because a model graded it; a check is evaluated in code. Tool calls that didn't run for real are marked **Stubbed**. A tool that ran for real shows the same status chip and request and response tabs as any other [transcript row](/docs/agents/live-conversations#request-and-response-details). **Show Turn** under a criterion highlights the turn the judge based it on. A test that repeated shows one **Attempt** button per attempt, each marked passed or failed, and opens on the first attempt that didn't pass. While a test is running, its row opens the run in progress, which says how many attempts are still running, and results appear as attempts finish. A test that hasn't run on the version opens with **Run Test**. The check and criterion wording comes from the test as it is now, so in **Run History** a test you edited after the run shows its new wording beside the old outcomes. In a narrow window the results cover the test list; **Esc** or the **×** closes them. **Run History** lists the version's earlier runs, each with how many attempts passed, such as "3 of 5 attempts passed". Select one to see every test in it, pick a test for its results, and select **All Tests** to go back to the list. ## Before you publish The publish dialog shows the version's test results. When every test passed, it says so. When tests are failing, stale, or running, when they haven't run, or when some of them didn't run on this version, it warns you and labels the confirm button **Publish Anyway**. It never stops you from publishing. The warning offers two buttons. **Run Tests** starts a run on the version and opens the Tests page. **View Tests** opens the Tests page without starting a run. While this version's tests are running, only **View Tests** appears. An agent runs one test run at a time, so if a run on another version is going, **Run Tests** says a run is already in progress and opens the Tests page without starting a new run. ## Test settings Select the gear beside the **Version** list: - **Simulated Caller Model** — plays the caller in simulated tests. - **Judge Model** — grades criteria against the transcript. - **Agent Temperature** — overrides the agent's temperature during tests only, from 0 to 1. A low value reduces variance between runs; it doesn't make them identical. OpenAI models, and Bedrock models with reasoning on, ignore it. Leave it blank to use the agent's own. - **Default Repeat** — how many times a test runs when it doesn't set its own **Repeat**. ## Limits and defaults | Item | Limit | | ----------------------------------- | --------------------------------------------------------- | | Tests per agent | 50 | | Name | 120 characters | | Description | 1,000 characters | | Caller messages in a scripted test | 40, each up to 4,000 characters; keypad entries count | | Keypad entry | 0-9, \*, #; up to 32 keys | | Contact attributes | 50; name 64 characters, value 500 | | Workflow variables | 20; name 64 characters, value 512 | | Caller memory | Prior Calls 1 to 10,000; each note up to 2,000 characters | | Persona | 4,000 characters | | Goal | 1,000 characters | | Max Turns in a simulated test | 1 to 40, default 20 | | Checks | 20 | | Criteria | 10, each up to 500 characters | | Text in **A reply contains** | 500 characters | | Regular expression | 200 characters | | Repeat and Default Repeat | 1 to 5, default 1 | | Simulated Caller Model, Judge Model | Default Claude Haiku 4.5 | | Runs and results | Kept for 90 days | | Suggested tests waiting per agent | New suggestions stop at 10 | | Tests per Recommend Tests request | Up to 3 | | Recent calls read per request | Up to 20 | ## Through the API Every action on this page is also available through the API. See [List an agent's tests](/docs/api/voice-agents/agent-tests/listAgentTests) and the other **Agent tests** operations in the API reference.