How AI outbound sales calls actually work in production

The answer, and the rule that decides whether you get to dial at all

On February 8, 2024 the FCC issued a Declaratory Ruling stating that AI-generated voices in calls are "artificial" voices under the Telephone Consumer Protection Act (FCC). That one sentence is the whole constraint.

So, directly: an AI outbound sales call is a voice agent that dials a number, holds a full-duplex conversation over a telephony carrier at human turn-taking speed, and ends in a booked meeting, a recorded disposition or a live transfer — and in the United States it is legally an artificial-voice call, which means you need prior express written consent on file before it rings a mobile phone (47 CFR 64.1200).

That ruling was not abstract. It followed the January 21, 2024 robocall that used a cloned Joe Biden voice to discourage voting in the New Hampshire primary. The FCC went on to issue a $6 million forfeiture in September 2024 against the political consultant who commissioned the calls, and Lingo Telecom, the carrier that originated the traffic, settled in August 2024 for $1 million plus a STIR/SHAKEN compliance plan (FCC Enforcement Bureau). The enforcement path runs through the originating carrier as well as the caller, which is why your telephony vendor will ask about consent before you ever ask them about price.

That is not a footnote you handle after you pick a vendor. It decides which lists exist for you, and that is the same question as whether this channel is a business or a lawsuit.

What the stack is actually made of

A voice agent is not a chatbot with a speaker. It is a real-time audio loop with a hard deadline on every turn.

The carrier leg is narrowband and brutal. Twilio's Media Streams ship audio over a WebSocket as base64-encoded mulaw/8000 payloads in 20 millisecond frames (Twilio docs) — 8-bit samples at an 8 kHz sample rate, which works out to 160 bytes per frame and a 64 kbps stream with an effective audio bandwidth of roughly 300–3,400 Hz. So whatever your TTS sounds like in the browser demo, the prospect hears it at 8 kHz through a phone speaker. Studio-grade voice cloning is money you set on fire on a PSTN call.

Turn detection is the part nobody budgets for. Silence-based voice activity detection fires on pauses, so the agent talks over someone who was mid-thought. Semantic turn detection — predicting whether an utterance is finished, not just whether audio stopped — is now exposed directly in the OpenAI Realtime API and shipped as a dedicated model in LiveKit Agents (OpenAI, LiveKit). It is the difference between a conversation and a kiosk.

Cascaded pipeline or speech-to-speech

Cascaded pipelines remain the default for regulated outbound, and the reason is evidentiary rather than aesthetic. On a regulated call the transcript is the artifact a compliance team has to produce on demand; in a speech-to-speech architecture the transcript arrives as a side-channel derived from the audio rather than as the exact token sequence the model emitted.

Speech-to-speech wins on latency and on nothing else. For a callback that has to feel instant, that is enough.

What a minute costs at published rates

Four meters run at once. Here is what the vendors actually print:

Work a three-minute connected call at those published rates and the floor is visible: 3 × $0.014 = $0.042 of carrier time, plus 3 × $0.05 = $0.15 of managed platform fee, so $0.192 before a single token of model or transcription spend. The carrier leg is the cheapest and most predictable line on the bill. The model leg is the variable one, and it scales with how long the agent talks, because output audio bills at double the input rate. A chatty system prompt is a cost line, not a style choice.

What the table will not tell you is your cost per outcome. A dial that hits voicemail still burns a carrier minute and an AMD charge. A 40-second "not interested" costs less than a 6-minute qualification that books nothing. Measure cost per booked meeting, not cost per minute, or you will optimise the agent toward being brief and useless.

Latency is the product

Humans tolerate roughly a beat of silence in conversation and then start talking again. The beat is measurable: a cross-linguistic study of ten languages found the median gap between conversational turns clusters around 200 milliseconds, with speakers treating gaps past about half a second as marked (Stivers et al., PNAS 2009). Every hop in the diagram above eats into that budget: carrier transit, turn detection, transcription, model time-to-first-token, TTS time-to-first-audio-byte, carrier transit back.

Track p95, not average. The average response is fine in every demo. The p95 is what happens when your enrichment API is slow and the agent goes silent for four seconds mid-sentence, and that is the turn where the prospect hangs up.

The specific killer is tool calls. The moment an agent looks up a calendar or a CRM record, the audio stream has nothing to send, and dead air on a phone call reads as a dropped connection. Decouple the two: emit a short acknowledgement token into the audio stream the instant the tool call starts, and run the lookup concurrently rather than blocking the turn. If a tool regularly takes more than a couple of seconds, it does not belong inside a live call. Pre-fetch it before the dial.

The failure modes that actually show up

• Unbounded loops. The agent re-asks the same qualifying question, or re-enters a clarification branch, and never terminates. The fix is a hard turn cap and a repeat-detector enforced outside the model, in the orchestration layer, with a defined exit action when the cap trips. A prompt instruction is not a cap. This is the same control-layer argument I make in agent orchestration is a control layer, not a framework.
• Voicemail treated as a human. Answering machine detection is probabilistic and adds detection latency before the agent may speak. Twilio's documented defaults make the cost concrete: a 2,400 ms speech threshold above which an utterance is classed as a machine greeting, a 1,200 ms speech-end threshold, a 5,000 ms initial silence timeout, and a 30-second overall MachineDetectionTimeout (Twilio). Agents that pitch into a voicemail greeting, or that pause awkwardly waiting for AMD to resolve, are both common. Decide explicitly what the agent does on machine_start versus human versus unknown.
• Barge-in that does not cut audio. If an interruption does not immediately stop the outbound TTS stream and flush the buffer, the prospect hears the agent plough on over them. With 20 ms frames already queued at the carrier, a flush that only stops generation still leaks audio. Interruption handling is a platform feature you test deliberately, not assume.
• Commitments the business cannot honour. A voice agent asked about pricing, contract terms or availability will produce a fluent answer. Constrain it to tool outputs and a refusal path for anything else.
• Transfer failure. The warm handoff to a human is the highest-value moment on the call and the least tested path. Test what happens when no rep picks up.
• Calling-hour and timezone bugs. The Telemarketing Sales Rule prohibits telemarketing calls before 8am or after 9pm at the called party's local time (16 CFR 310.4(c)). An agent that dials off a UTC timestamp against a list with missing timezone data will eventually dial someone at 7am local.

Compliance is a dialing precondition

Four things gate the list, not the script.

Disclose on the first turn of every call and stop maintaining a state-by-state matrix — the matrix is stale the week you finish it. On consent, the obligation is evidentiary: if you cannot produce the record for a given call ID, you do not have consent. And burning through rotating numbers is how you get labeled "Spam Likely" and lose the channel entirely, because full A-attestation depends on the provider having a verified relationship with the number you are presenting.

What I would not use

I would not point an AI voice agent at a cold purchased list. Not because the technology cannot do it — it can, trivially, which is the problem — but because the economics only work at volumes that put you squarely inside artificial-voice robocall liability, and because the channel degrades for everyone the moment carriers start labeling your numbers. The cost of being wrong is $500 per violation, trebled to $1,500 for willful or knowing violations, with a private right of action and no cap on class size (47 U.S.C. § 227(b)(3)). At 10,000 non-consented dials that is a $5 million exposure before anyone argues willfulness. That is an asymmetric bet against yourself.

I would not give a voice agent authority over discounts, pricing exceptions or anything that moves money. Qualify, book, transfer, disposition. That is the job.

And I would not build voice before the data layer works. A voice agent calling off a list with 30% stale phone numbers and no timezone field is not an outbound motion, it is an expensive way to generate wrong-number dispositions. Fix the list first — an agent that qualifies leads from a CSV is a smaller build and tells you whether the data is good enough to dial.

Where it genuinely earns its keep

Inbound callback is the case that works. Someone fills a form, the agent dials back in seconds, consent is fresh and documented, and the only variable being optimised is response time — the one place a machine beats a human rep outright. The payoff is well documented: an analysis of some 2,241 US companies found firms that responded to a web lead within an hour were nearly seven times more likely to have a meaningful conversation with a decision maker than those that waited an hour longer, and more than 60 times more likely than those that waited 24 hours (HBR, 2011). Collapsing that window from minutes to seconds is a latency problem, which is exactly what this stack is for.

After that: opted-in re-engagement with existing customers, trial users and people who asked to be contacted. Then operational calls — no-show follow-up, renewal reminders, scheduling confirmation. Low ambiguity, high completion rate, no pitch. Pre-dial qualification also holds up, where the agent confirms the number is live and reaches the right person, then hands a shortlist to humans.

None of that is "AI replaces SDRs." I wrote about where that framing breaks in what an AI SDR agent is, and what it is not. Voice is one channel inside a motion, and it carries the highest regulatory surface area and the tightest latency budget of any of them. Build it last.

What you log, per call

If the vendor dashboard is your only record, you cannot debug and you cannot defend. The per-call record needs: audio recording and transcript, consent record ID and source, turn-level latency at p50 and p95, interruption count, every tool call with arguments and result, the disposition, and whether a turn cap or guardrail fired.

Retention follows the exposure, not the dashboard's default. TCPA claims run on the general federal four-year statute of limitations (28 U.S.C. § 1658), so a consent record that expires with your vendor's 30-day or 90-day log window is not a defence.

Build or buy comes down to one question: whose storage holds that record. Managed platforms like Vapi charge a per-minute fee on top of pass-through costs and hand you their logging (Vapi). Self-hosting on Pipecat or LiveKit Agents means you own the pipeline, the latency tuning and the log schema, and you also own the pager (Pipecat, LiveKit).

Start managed. Move when the logging becomes the reason you cannot ship. The test for whether you are ready to dial at volume is simple and unforgiving: pick a random call ID from yesterday and produce the consent record, the full transcript and the p95 turn latency for that call in under a minute. If you cannot, you are not running an outbound motion. You are running an experiment with someone else's phone number.