- Structure every prompt the same way: role, goal, tone, knowledge, flow, tools, hand-offs and rules.
- Write for the ear: one or two short sentences per turn, no lists, no URLs, no long numbers read in one go.
- Tell the agent exactly when and how to use each tool, and what to do when a tool fails.
- Guardrails and hand-off rules are as important as the happy path.
- Test with realistic, messy calls and change one thing at a time.
The system prompt is the instruction manual for your AI voice agent. It decides how the agent greets callers, what it asks, what it knows, when it books and when it hands off. Most voice agents that sound robotic, ramble or make things up have a prompt problem, not a technology problem. This guide covers how to write prompts that work on real phone calls.
Why voice prompts are different from chat prompts
| Chat | Voice | |
|---|---|---|
| Reply length | A paragraph is fine | One or two short sentences |
| Lists and links | Easy to read | Impossible to follow by ear |
| Interruptions | Rare | Constant; the agent must stop and listen |
| Numbers and spellings | Shown exactly | Must be read slowly and confirmed |
| Silence | Normal | Feels broken after a second or two |
Everything in a voice prompt should push the agent toward short, conversational turns that end with a clear question or confirmation.
The structure we use
We write every production prompt in the same eight sections. Consistent structure makes prompts easier to debug and update.
- Role: who the agent is and which business it represents.
- Goal: the single outcome it works toward on every call.
- Tone and speech style: personality and rules for speaking.
- Knowledge: the facts it may use, and what to do when it doesn’t know.
- Conversation flow: the steps of a typical call.
- Tools: when and how to use each one.
- Hand-off rules: when to transfer or take a message.
- Guardrails: what it must never do.
Our free AI agent prompt generator produces a draft in exactly this structure from a few details about your business.
1. Role and goal
Be specific. “You are a helpful assistant” gives the model nothing to work with. Compare:
2. Tone and speech style
Describe the personality, then add concrete speaking rules:
- Speak in short sentences; never more than two before pausing.
- Ask one question at a time.
- Never read lists of more than two options; offer two times, not five.
- Don’t read URLs or email addresses aloud; offer to text them instead.
- Read phone numbers in groups and confirm them back.
- Use the caller’s name once you know it, but not in every sentence.
3. Knowledge and honesty
List the facts the agent may use, such as services, hours, service area and price ranges, or point it to a knowledge base. Then add the most important rule in any business prompt: if the answer isn’t in your knowledge, say you don’t know and offer to take a message. Agents that guess prices or policies create real problems.
4. Conversation flow
Write the typical call as numbered steps, but tell the agent to adapt: skip questions already answered, and follow the caller if they jump ahead. For example: greet, understand the need, collect address and urgency, offer two available times, confirm details, explain what happens next.
5. Tool instructions
Tools are where agents most often go wrong. For each tool, state:
- When to call it: “Call check_availability only after you know the service type and the caller’s preferred day.”
- What to say while waiting: “Say ‘Let me check that for you’ before calling the tool.”
- How to use the result: “Offer the two earliest times returned.”
- What to do on failure: “If the tool returns an error or no slots, apologize and offer to take a message for a callback.”
6. Hand-off rules
Be explicit about when a human takes over: emergencies, safety issues, complaints, legal or medical questions, pricing exceptions and any direct request for a person. Specify the transfer number and what to say before transferring, and what to do if nobody answers.
7. Guardrails
- Answer honestly if asked whether you’re an AI.
- Never give medical, legal or financial advice.
- Never collect card numbers or sensitive data without a secure process.
- Never promise discounts, refunds or exceptions.
- Stay on topic; politely redirect unrelated requests.
Common prompt mistakes
| Mistake | What happens | Fix |
|---|---|---|
| Prompt is a wall of text | The agent ignores important rules | Use clear sections and short bullet rules |
| No “I don’t know” rule | The agent invents prices or policies | Explicitly require it to admit gaps |
| Lists read aloud | Callers get lost | Limit options to two per turn |
| Vague tool instructions | Tools are called too early or not at all | Define timing, inputs and failure handling |
| Long greeting | Callers interrupt or hang up | Keep it under about ten words |
| Changing many things at once | Impossible to know what helped | Change one section per test round |
Testing your prompt
Run scripted calls covering the happy path, interruptions, mumbled answers, off-topic questions, angry callers and tool failures. Score each call against your goal and note exactly where it went wrong. Change one thing, re-run the same calls and compare. For more on the full build process, see how to build an AI voice agent, and for ready-made examples, our voice agent script templates.
Platform notes
The principles apply on every platform, but details differ. On Retell AI, conversation-flow agents let you split long prompts into nodes, which helps with predictable calls. On Vapi, tools and multi-assistant setups let you keep each prompt focused on one job. In GoHighLevel, keep prompts concise and lean on your knowledge base for facts.
Written by Abdul Moeez, AI Automation Expert
Abdul designs and builds the AI voice agents, chatbots and automation systems Voxil AI ships: from conversation design and integrations to testing on real calls.