All Nine of My Voice Agents Are One Giant Prompt. Here Is Where That Stops Working.

This week I went through every voice agent config I have exported on disk. Nine client agents: HVAC, roofing, dental, water treatment, two law firms, credit repair, debt settlement, an appointment setter. Every single one is the same architecture. One prompt. No states, no flow graph. The prompts run from about 2,400 words at the small end to about 4,400 words for a personal injury intake line, and each agent has three or four tools.

Retell, the platform I build on, gives you three ways to structure an agent: a single prompt, a multi-prompt agent split into states, and a node-based conversation flow. I picked single prompt for all nine. I still think that was mostly right. But reading them side by side showed me something I had not admitted: most of those prompts are already state machines. I just wrote the states in prose.

This post is about where the line is, and what the failures look like when you are on the wrong side of it.

The three shapes, in developer terms

Single prompt. One block of instructions holds identity, scope, the call steps, the knowledge, escalation rules and tool usage. The model reads all of it on every turn and decides what to do next. Think of it as one big function with no explicit control flow, where the model is the interpreter.

Multi-prompt. The agent is split into states, each with its own short prompt and its own tool list, plus transition rules between them. This is a state machine where each state's body is still natural language, but the edges are yours.

Conversation flow. A visual graph. Each node is a step: say something, ask something, call a function, branch on a condition, transfer. The call follows the path you drew. This is the closest thing to deterministic code you get on the platform, with all the rigidity that implies.

What my "single" prompts actually look like

Here is the heading structure of the injury lawyer agent, the 4,400 word one:

## Role and Objective
## Personality
## Context
## Instructions
### Identity Rules - CRITICAL
### Communication Rules
### Scheduling Rules
### Urgent Escalation Rules
## Stages
### Stage 1: Greeting
### Stage 2: Case Type Confirmation
### Stage 3: Time Qualification
### Stage 4: Injury and Urgency Assessment
### Stage 5: Qualification Decision
### Stage 6: Booking
### Stage 7: Post-Booking
## Objection Handling
## Tools
## Example Interactions

Seven numbered stages. That is a multi-prompt agent wearing a trench coat. The difference is that the model sees all seven stages, all the tools and all the rules on every turn, and nothing enforces that it is in Stage 3 when it thinks it is in Stage 3.

That works surprisingly well, as long as the rules never disagree with each other. Which brings me to the bug I learned the most from.

A rule in three places

One of the HVAC agents has two versions in my folder, the original and one I named FIXED. The diff is small, and almost all of it is about one tool.

The agent has a tool called getWeekAvailability that loads the next two weeks of open slots. The plan was simple: at the start of the scheduling stage, call it silently, then ask the caller what day works. The original Stage 4 said exactly that:

1. At the START of this stage, BEFORE asking anything,
   silently call getWeekAvailability ONCE. Do not announce this.
2. "Thanks for confirming. What day works best for you,
   and do you prefer a morning or afternoon window?"

Further down, in the Tools section, a general rule:

- Before any tool, use a brief verbal bridge:
  "Let me check on that," "One moment," or "Just a moment."

And further down again, in the example conversations, the agent asks "What day works best for you?" first, and the tool call happens after the question.

So the same behavior was specified three times, in three sections, and the three disagreed. "Silently, before asking" in the stage. "Say a bridge phrase before any tool" in the tool rules. "Ask first, then call" in the examples. The model has no way to know which one is the spec and which ones are stale. Like anyone handed contradictory docs, it picks one per call, and not always the same one.

The fix touched all three places. The stage now says to call the tool immediately and say nothing before it. The bridge rule became "before checkCalendar or bookCalendar", with an explicit "Do NOT use a verbal bridge before getWeekAvailability". Every example was rewritten so the tool call comes first and the question comes after it.

The lesson I took from it is a code lesson, not a prompt lesson. In a single prompt there is no scoping. A rule written for "any tool" applies everywhere, forever, including to the tool you add later. Every general rule is a global variable, and every example conversation is a second copy of the logic that can drift from the first. I wrote about the same problem from another angle in a prompt with placeholders being a function signature nobody type-checks: prompts have all the failure modes of code and none of the tooling.

In a multi-prompt agent, that bridge rule would have lived in the states that use the calendar tools, and the availability tool would have lived in a state where the rule simply did not exist.

So why are all nine still single prompts?

Because the cost of splitting is real, and for most of these calls it is not worth paying yet.

Single prompts are the most natural sounding of the three. Callers do not follow your stages. They give their address while you are asking about the problem, ask about price during booking, or answer two questions at once. A single prompt handles that because nothing stops the model from jumping around. In a multi-prompt agent, every one of those jumps is a transition you have to design, and a caller who asks a pricing question while the agent sits in the booking state can get a blank stare. In a flow, an answer no edge expected is a dead end.

Single prompts are also the easiest to change. When I am editing an agent several times a week based on real calls, a flat prompt I can read top to bottom beats a graph where one changed node quietly breaks a path three nodes later.

Where I draw the line now

After the side by side read, these are the signals I actually use.

The script is not negotiable. If a client needs every caller to hear a disclosure before any questions, or a sales leader signed off on exact qualification wording, that is a conversation flow. A single prompt will follow a script most of the time. Compliance needs all of the time.

The call does several jobs in sequence. Verify identity, then look up a record, then book or reschedule, then collect details for a follow up text. Each of those is a job with its own tools and its own failure modes. That is when multi-prompt pays off, because each state gets a short prompt you can test on its own.

The tool list grows. My agents sit at three or four tools, and the calendar ones are already where my one real ordering bug lived. Somewhere past that, the model starts reaching for the wrong function at the wrong moment. Scoping tools to states fixes most of it.

You keep writing "CRITICAL". My injury lawyer prompt has a section literally titled "Identity Rules - CRITICAL". Shouting in a prompt is a sign that a rule is fighting with other rules for attention. One or two of those is fine. When you need many, the prompt is asking to be split.

Your stages have numbers. If you have already written Stage 1 through Stage 7, you have designed the state machine. The only question left is whether you want the platform to enforce it.

Rules I follow inside a single prompt

Since most of my agents will stay single prompts for a while, these are the habits that came out of the HVAC fix:

1. Say each rule once. If a behavior is described in the stage, the tool description and the examples, change all three or delete two of them.
2. Scope general rules by name. "Before checkCalendar or bookCalendar" instead of "before any tool". New tools should not inherit old rules by accident.
3. Treat examples as tests that can be wrong. In my experience the model leans on examples at least as hard as on instructions, so an outdated example is a bug, not documentation.
4. Diff before you ship. I keep the previous version next to the new one for exactly this reason. Testing a live line is its own problem, covered in there is no repro for a phone call.

The client never sees the architecture

None of my clients know or care whether their receptionist is one prompt or a twenty node graph. They care whether calls were answered and appointments got booked. That is part of why we built VoiceDash, the white-label portal where each client sees their own Retell calls, transcripts and analytics. When an agent moves from one shape to another, the client's view does not change.

Transcripts are also how I spot when an agent has outgrown its shape: the same skipped step or the same stall after a topic change, across several calls. That is the signal to split, not the word count.

The original, more agency-focused version of this guide lives on the VoiceDash blog.

If you build LLM agents with long prompts, voice or not: where did you draw the line between one prompt and explicit states, and what finally pushed you over it?