Skip to content
Back to Blog
Optimise Your Voice Agent – Boost Performance and Improve AI Telephony
OptimisationPerformanceVoice AgentNovember 2, 20258 min

Optimise Your Voice Agent – Boost Performance and Improve AI Telephony

A Voice Agent that is just as good — or just as poor — after three months as it was on day one is a bad Voice Agent. The best deployments improve continuously — because a systematic optimisation process has been established that translates data into insights and insights into better configurations. This article shows you the optimisation methodology that successful SMEs apply.

The Fundamental Problem: Optimisation Without Method

Many companies "optimise" reactively: a customer complains → they try to fix the specific problem → they hope a similar complaint won't arise again. This is not an optimisation process — it is firefighting.

Systematic optimisation works differently: you continuously collect data, identify patterns, formulate hypotheses, test changes and measure the impact. This cycle — known as PDCA (Plan-Do-Check-Act) or the Lean Improvement Cycle — is the backbone of professional Voice Agent optimisation.

Teams with a systematic process find and fix problems earlier than teams that only react to complaints. The voice agent KPIs guide lists the metrics to track.

Phase 1: Collect Data – What You Really Need to Measure

The Data Foundation

Optimisation without data is guesswork. The most important data sources for Voice Agent optimisation:

Conversation transcripts: The richest source (with anicall.io every call is transcribed and summarised and stored searchably in the dashboard). Every conversation is a data point about what customers want, how they phrase it, where misunderstandings arise and where the agent falls short.

Intent confidence scores: How confident was the agent in its intent classification? Low confidence scores signal training gaps.

Conversation drop-off points: Where does the conversation break off or get escalated? These points are optimisation priorities.

Post-call CSAT: Customer satisfaction immediately after the conversation. Best collected automatically via SMS or a short prompt at the end of the call.

CRM outcome data: Was the business objective achieved? (Appointment booked, problem solved, conversion achieved?)

Ensuring Data Quality

Poor data leads to wrong conclusions. Check:

  • Are all conversation transcripts captured completely?
  • Is CSAT data collected for all use cases?
  • Are there data gaps (certain times of day, call types that are not captured)?
  • Are integrations correct and complete?

Phase 2: Analyse – Recognising Patterns

The Five Analytical Lenses

Lens 1 – Volume analysis: Which use cases and intent categories have the highest volume? That is where optimisation is most worthwhile.

Lens 2 – Failure mode analysis: What are the most common error types? Ranked by frequency and impact.

Lens 3 – Funnel analysis: What does the conversation flow look like? At which point do customers leave the process prematurely?

Lens 4 – Segment analysis: Are there customer segments that systematically produce worse results? (Older customers, certain regions, certain call times?)

Lens 5 – Own baseline: How do your metrics compare with your first four weeks and with your human-handled calls?

The Root Cause Analysis Method

When you have identified an anomaly (e.g. high escalation rate for a particular intent), proceed systematically:

  1. Collect data: All conversations with this intent in the last 30 days
  2. Identify patterns: What do the conversations have in common? Particular phrases? Particular conversation phases?
  3. Formulate hypothesis: "The high escalation rate arises because the agent does not process X correctly"
  4. Test hypothesis: A/B test with adjusted configuration

Phase 3: Formulate and Prioritise Hypotheses

The Hypothesis Framework

Good optimisation hypotheses follow this pattern:

"We believe that [change X] will improve [metric Y] by [expected effect Z], because [reasoning/evidence]."

Example: "We believe that adding a proactive appointment alternative ('If Tuesday doesn't work, we also have availability on Thursday') will improve the appointment conversion rate, because our call data shows that many customers hang up after the first appointment offer without asking for an alternative." (Fill in your own measured numbers.)

This structure forces evidence-based thinking and makes measuring results clear.

Prioritisation Matrix

Not all hypotheses are equally valuable. Evaluate on two axes:

Impact: How large is the expected effect? (Low/Medium/High) Effort: How complex is the implementation? (Low/Medium/High)

Prioritise: High impact + Low effort = Quick wins (address immediately). High impact + High effort = Strategic projects (plan and budget). Low impact = low priority.

Phase 4: A/B Testing – Validating Changes

Why A/B Testing and Not Just Making the Change?

The natural reaction: "If we know what the problem is, we just change it." The risk: a change that helps in one situation can harm another. Without A/B testing, you don't know whether an improvement is due to your change or to other factors.

A/B Testing for Voice Agents: Practical Implementation

Variant A: Existing configuration (control group) Variant B: New configuration (test group)

Traffic split: 50/50 for equal sample sizes, or 80/20 if you want to minimise the risk of a deterioration.

Metrics for the test:

  • Primary metric: What you want to improve (e.g. Conversion Rate)
  • Guardrail metrics: What must not get worse (e.g. CSAT, Escalation Rate)

Duration: At least 7 days, ideally 14 days, to account for day-of-week effects. Minimum sample size: 100 conversations per variant for statistically robust results.

Interpreting Results

  • Variant B is statistically significantly better (p < 0.05) on primary metric with no guardrail violation → Roll out
  • No statistically significant difference → Hypothesis rejected, formulate new hypothesis
  • Variant B improves primary metric but worsens a guardrail → Revision needed

Phase 5: Prompt Optimisation in Detail

The Most Common Prompt Problems

Problem 1: Instructions too vague Symptom: Agent behaves inconsistently, sometimes correctly, sometimes incorrectly. Solution: More explicit if-then rules, concrete examples in the prompt.

Problem 2: Too many instructions at once Symptom: Agent "forgets" instructions or prioritises incorrectly. Solution: Modularise prompts, set priorities explicitly.

Problem 3: Missing fallback definitions Symptom: Agent gives useless or incorrect answers for unknown requests. Solution: Explicit out-of-scope handling instructions with a clear escalation path. In anicall.io, unanswered questions appear as knowledge gaps in the dashboard; a human answers or dismisses them, and approved answers are used from the next call on. A second model also checks every answer against its sources after the call (traffic light: green, yellow, red). See AI phone assistant without hallucinations.

Problem 4: Static persona without emotional adaptability Symptom: Agent sounds robotic in emotional conversations. Solution: Sentiment detection hooks and different conversation modes (neutral/empathetic/solution-oriented).

Prompt Optimisation Workflow

  1. Identify conversations with low scores
  2. Locate the section of the conversation causing the problem
  3. Analyse the current prompt for that section
  4. Formulate an optimised prompt
  5. Set up an A/B test
  6. Evaluate after 14 days
  7. Roll out or iterate

Phase 6: Conversion Funnel Optimisation

The Voice Agent Conversion Funnel

For outbound Voice Agents, conversion optimisation is particularly relevant:

Step 1 – Connection Rate: How many of the dialled numbers are actually answered? Optimisation: Optimise call timing (day of week, time of day, season).

Step 2 – Engagement Rate: How many of those who connect listen for longer than 20 seconds? Optimisation: The conversation opener — the first 10 seconds are decisive.

Step 3 – Intent Qualification: How many of those engaged have a relevant concern/interest? Optimisation: Improve targeting, work with more qualified lists.

Step 4 – Desired Action Rate: How many book an appointment, give consent, place an order, etc.? Optimisation: Value proposition, offer framing, urgency elements.

Step 5 – Post-Call Follow-Through: Do those who booked actually show up for the appointment? Optimisation: Reminder sequences, confirmation SMS, re-communicating the value of the appointment.

Latency Optimisation: Speed as a Quality Feature

In a phone call, long pauses before the agent answers feel unnatural, and callers quickly grow impatient or talk over the agent. Measure the pause between the end of the caller's sentence and the start of the answer in your own test calls. The voice agent architecture guide explains where the time goes.

Latency sources and optimisation measures:

ASR latency (speech recognition): Depends on the provider and is barely optimisable by the user.

LLM and prompt processing time: Optimisable through more efficient prompts and shorter context.

API call latency: Variable, from milliseconds to seconds. Optimisable through caching of frequent requests and regional server locations.

TTS latency (speech output): Optimisable through TTS streaming rather than full pre-generation.

Measuring Optimisation ROI

Every optimisation measure should demonstrate its ROI. Calculation:

ROI of an optimisation measure:

  • Effort: Internal hours × hourly rate + provider costs for adjustment
  • Benefit: Delta in primary metric × monetisation factor

Hypothetical example: Optimising the conversation opener increases the conversion rate from 18% to 23% with 500 outbound calls/month. Average order value: €400.

  • Additional conversions: 500 × 5% = 25/month = 300/year
  • Additional revenue: 300 × €400 = €120,000/year
  • Optimisation effort: 8 hours × €80 = €640
  • ROI: 18,650%

This example is extreme — but it illustrates why conversion optimisation for Voice Agents is among the most profitable measures available.

Conclusion: Optimisation as a Core Discipline

A Voice Agent is not a static product — it is a learning system that benefits from systematic human guidance. Companies that establish optimisation as a permanent practice build a lead over time: their agent keeps improving while competitors stagnate with their initial configuration. If you are still choosing a setup, start with voice agent testing.


Put it into practice: review the transcripts, close the knowledge gaps, repeat.

Test your agent for free in the browser at anicall.io and see the AI phone assistant for how reviews and knowledge gaps work.