A technical deep-dive into Tunk.ai's voice agent platform — covering real-time WebSocket audio pipeline, LLM orchestration, batch calling at scale, sub-second latency, and autoscalable AWS ECS deployment.
Tunk.ai is a production-grade Voice AI Agent platform for enterprise contact centres and customer experience teams. It automates inbound and outbound voice calls using a globally connected WebSocket architecture — handling real-time audio streaming, LLM reasoning, live API integrations, and batch call orchestration at any scale.
A global WebSocket server connects Tunk.ai to any telephony platform. Audio streams in real-time — no batch uploads, no polling. Every byte of customer speech is streamed continuously for the lowest possible latency.
Multi-provider LLM support (OpenAI, Gemini) with RAG knowledge grounding, tool-calling, and safety guardrails — all orchestrated within a single conversational turn under 300ms of reasoning time.
Schedule, run, pause, resume, and terminate outbound call batches of any size. Built on Redis Bull MQ for durable job management with automatic retry logic and real-time progress tracking.
Every live call runs through a streaming WebSocket pipeline with six distinct stages — each optimised for minimum latency. Audio is never batched: it streams continuously from the customer's phone to our servers, through AI processing, and back to the caller.
Tunk.ai exposes a global WebSocket endpoint that telephony platforms (Plivo, Twilio, and others) connect to when a call is established. The customer's phone audio is captured at the dialer level and streamed to our servers in real-time binary streaming packets. This WebSocket handshake happens once per call; audio then flows continuously without reconnection overhead.
Our STT engine layer maintains a persistent WebSocket connection to the selected STT provider (Deepgram, Smallest.ai, Azure Speech, AssemblyAI, or others). Incoming audio packets are forwarded in real-time. The provider begins returning partial transcription results as the customer speaks — enabling the LLM pipeline to begin processing before the utterance ends. Final transcription arrives within 100–300ms depending on provider response speed.
The transcribed utterance — together with the conversation history, system prompt rules, and any retrieved Knowledge Base context — is sent to the LLM (OpenAI or Gemini). The LLM acts as the agent's brain: understanding the spoken input, maintaining dialogue state, and formulating a natural response. Simultaneously, if a Knowledge Base lookup or Tool/API/MCP call is needed, those run in parallel during this stage. Response tokens stream back from the LLM immediately, fed directly to the TTS pipeline without waiting for the full response.
LLM response text is streamed token-by-token to the TTS provider over a dedicated WebSocket connection. TTS synthesis begins the moment the first sentence is complete — not after the entire response is generated. The provider streams synthesised audio packets back to Tunk.ai as they are generated, achieving a low Time-To-First-Byte (TTFB) of 150–200ms on leading providers. Total audio generation takes 300–500ms depending on the provider's streaming speed and response length.
Synthesised audio packets are transmitted in real-time back to the telephony platform over the existing WebSocket connection. The dialer plays audio to the customer's phone as it arrives — there is no buffering of the full audio before playback begins. Network transmission adds 10–20ms. The customer hears the agent's response almost immediately after the last TTS byte arrives.
The moment a call ends, Tunk.ai kicks off a fully automated async processing pipeline. Every deliverable — recording, analytics, summary, CRM update — is ready within 5–10 seconds of call termination, with no manual intervention.
The LLM analyses the full call transcript and classifies the final outcome (disposition) — e.g. booking_confirmed, payment_failed, callback_requested, or any custom outcome defined by the client.
Structured extraction of any information spoken by the user — names, dates, reference numbers, preferences, complaints — mapped to client-defined schema fields and pushed to the CRM or webhook endpoint.
Utterance-level and call-level sentiment scoring alongside a concise 2–3 sentence call summary — both generated automatically and pushed to the client's dashboard and CRM within seconds.
Tunk.ai provides a global WebSocket server that any WebSocket-capable telephony platform can connect to — enabling real-time bidirectional audio streaming without polling, buffering, or custom SDK installation.
| Platform | Integration Type | Status |
|---|---|---|
| Plivo | WebSocket Media Stream | ● Live |
| Twilio | Media Streams (WSS) | ● Live |
| Vonage | WebSocket Audio | ● Coming Soon |
| CarrierX | SIP + WebSocket Bridge | ● Coming Soon |
| Custom SIP | SIP + WebRTC Bridge | ● Coming Soon |
When an inbound or outbound call connects, the telephony platform initiates a WebSocket connection to Tunk.ai:
Tunk.ai is provider-agnostic — the optimal STT and TTS model is selected per client based on language requirements, latency targets, and voice quality preferences. Providers can be mixed per agent or per call leg.
| Provider | Model(s) | Latency (TTFT) | Languages | Streaming | Speciality |
|---|---|---|---|---|---|
| Deepgram | Nova-2 · Nova-3 | ~100ms | 30+ | ✓ WS | Lowest latency, phone-optimised |
| Smallest.ai | Lightning · Pro | ~80ms | 20+ | ✓ WS | Ultra-low latency, edge-optimised |
| Azure Speech | Fast · Accurate | ~150ms | 140+ | ✓ WS | Enterprise multilingual, compliance |
| AssemblyAI | universal-2 · universal-3.5-pro | ~150ms | 17+ | ✓ WS | High accuracy, turn detection, diarization |
| Google STT | Chirp · Chirp 2 | ~200ms | 125+ | ✓ gRPC | Broadest language coverage |
| OpenAI Realtime | gpt-realtime-whisper | ~200ms | 57+ | ✓ WS | Tunable latency, VAD-aware, noisy environments |
| Provider | Model(s) | TTFB | Languages | Voice Clone | SSML | Speciality |
|---|---|---|---|---|---|---|
| ElevenLabs | Turbo v2.5 · Flash | ~180ms | 32+ | ✓ Instant | ✓ | Most realistic, branded voices |
| Deepgram Aura | Aura-2 · Asteria | ~120ms | 15+ | — | ✓ | Lowest latency at scale |
| Smallest.ai | Lightning TTS | ~90ms | 10+ | ✓ | Partial | Ultra-fast, sub-100ms TTFB |
| Azure Neural TTS | Neural · HD | ~140ms | 140+ | ✓ Custom | ✓ Full | Enterprise SSML, widest language |
| Google Cloud TTS | WaveNet · Studio | ~160ms | 50+ | — | ✓ Full | Regulated sectors, multilingual |
| Amazon Polly | Neural · Generative | ~150ms | 30+ | — | ✓ Full | AWS-native, low cost at volume |
| Cartesia | Sonic-3 · Sonic-3.5 | ~80ms | 15+ | ✓ Instant | Partial | Ultra-low TTFB, sub-100ms first byte |
Language is auto-detected from the caller's first utterance — TTS voice and LLM system prompt switch dynamically:
Fine-grained control over the agent's spoken delivery via SSML:
Every Tunk.ai agent is grounded in client-specific business data through a Retrieval-Augmented Generation (RAG) pipeline. This eliminates hallucinations without fine-tuning the base LLM — agents answer from your documents, not from model priors.
Documents are automatically chunked, embedded, and indexed. Updates to any document propagate to the live agent in under 5 minutes with no downtime.
Milvus stores high-dimensional embeddings with HNSW indexing for sub-millisecond approximate nearest-neighbour search across millions of document chunks. Pinecone supported as alternative.
Tunk.ai's batch calling engine lets you schedule, execute, monitor, and control outbound call campaigns at any scale — from a few dozen calls to millions. Built on Redis Bull MQ for durable, fault-tolerant job management with automatic retry logic.
| Operation | Behaviour |
|---|---|
| 🟢 Schedule | Create a batch with start time, concurrency limit, calling window, and contact list |
| ▶️ Run | Dispatch jobs from queue to worker pool; workers auto-scale with call volume |
| ⏸ Pause | Halt new job dispatch; active calls complete gracefully; state preserved in Redis |
| ▶️ Resume | Continue dispatch from exact position; no duplicate calls; no data loss |
| 🛑 Terminate | Cancel all pending jobs; active calls complete naturally; results flushed to DB |
Failed calls (no answer, busy, network error) are automatically retried with configurable backoff:
Tunk.ai agents can read and write live data across your entire tech stack during the call. Tool calling is natively wired into the LLM reasoning loop — the model decides when to call an API and what to do with the result.
Fetch live customer data mid-call: account balance, order status, booking availability, CRM fields — enabling hyper-personalised responses without pre-scripting every scenario.
Create bookings, update CRM records, raise support tickets, trigger payment flows, send SMS confirmations, push call dispositions — all within the live conversation.
Model Context Protocol allows any third-party service to be connected as a tool without custom integration code — dramatically reducing new client onboarding time.
When the LLM detects that a situation requires human intervention — or the customer explicitly requests it and grants permission — the agent seamlessly transfers the live call to a human agent without dropping the connection. Transfer can be triggered autonomously by the agent or manually by the caller on the call.
Anchor is Tunk.ai's chat-based agent orchestration interface. It lets non-technical users design, configure, and deploy fully functional voice AI agents in minutes — through a natural conversation, not forms or code.
Describe your use case in plain English — Anchor configures the agent's system prompt, selects the optimal STT/TTS providers, links your knowledge base, and connects your telephony number automatically.
Anchor exposes the full agent config — voice selection, system prompt rules, retry settings, tool connections, knowledge base uploads, and batch campaign parameters — all via natural language or direct UI.
Sensitive actions — telephony changes, agent deletion, new integrations — require explicit user approval before execution. Anchor never makes destructive changes autonomously.
Tunk.ai runs on a fully containerised, cloud-native infrastructure on Amazon Web Services. The platform autoscales horizontally in response to inbound call volume — adding capacity without operator intervention or service disruption.
Defence-in-depth across every layer — from WebSocket ingress to long-term archival storage.
| Layer | Control | Standard | Detail |
|---|---|---|---|
| Transport | TLS 1.3 only | RFC 8446 | All APIs, webhooks, WebSocket streams |
| Storage | AES-256 encryption | FIPS 197 | S3, RDS, Redis (EC2), Vector DB |
| Secrets | AWS Secrets Manager | KMS + IAM | Zero plaintext secrets in env or code |
| Network | VPC + Security Groups | AWS VPC | Private subnets, no public ECS IPs |
| Identity | RBAC + MFA | OWASP IAM | Short-lived tokens, all admin requires MFA |
| Webhooks | HMAC-SHA256 signing | RFC 2104 | Replay protection via nonce + timestamp |
| Privacy | GDPR DPAs available | GDPR Art.28 | Right-to-erasure supported, data residency options |
Requirements gathering, telephony platform selection, DID number provisioning, and success KPI definition. Anchor account configured and team access provisioned.
Agent built via Anchor: system prompt, voice selection, knowledge base ingestion, and CRM/API tool connections configured. WebSocket link to telephony platform established.
Internal sandbox testing across happy paths, edge cases, and adversarial inputs. Conversation flow refined via Anchor. Latency verified within acceptable range.
Limited live rollout (10–20% of call volume) with real-time post-call analytics monitoring. Rapid iteration based on first live call data — most agents are production-ready within hours.
Full production rollout across all call volume. Ongoing monitoring, continuous agent optimisation, and monthly performance reviews.
Full DTMF tone detection enabling agents to navigate legacy IVR trees, accept PIN inputs, and route calls through existing telephony infrastructure without system replacement.
Direct native integrations with Vonage and CarrierX, alongside Custom SIP trunking capabilities to connect any enterprise PBX directly into Tunk.ai's high-speed WebSocket pipeline.
Full adoption of the IETF vCon (Virtual Conversation) open standard — packaging call recordings, multi-channel transcripts, AI analysis, metadata, and cryptographic consent records into standardized JSON containers for seamless enterprise data portability and compliance.
Orchestrate specialized sub-agents working in tandem within a single call. A primary triage agent can dynamically transfer control to specialized sub-agents (e.g. billing, technical support, booking) mid-call, resolving complex multi-domain workflows seamlessly.
Enable a single agent instance to dynamically switch languages mid-conversation while executing dedicated, language-customized system prompts and cultural guidelines for each supported dialect.
| Term | Definition |
|---|---|
| LLM | Large Language Model — the AI engine powering intent detection, dialogue management, and response generation (OpenAI, Gemini) |
| RAG | Retrieval-Augmented Generation — injecting retrieved knowledge base chunks into the LLM prompt to ground responses in factual, client-specific data |
| STT | Speech Recognition — conversion of streaming audio into text transcript via WebSocket connection to providers like Deepgram, Smallest.ai, Azure, or AssemblyAI |
| TTS | Text-to-Speech — synthesis of LLM-generated text into human-like audio via WebSocket to providers like ElevenLabs, Deepgram Aura, Azure, Google, or Amazon Polly |
| Bull MQ | Redis-backed durable job queue library used for batch call scheduling, execution, pause/resume, and retry management |
| VAD | Voice Activity Detection — detects when a caller is speaking vs. silent; used for utterance segmentation and barge-in interruption |
| Disposition | The final outcome classification of a completed call — e.g. booking_confirmed, payment_failed, callback_requested, escalated_to_human |
| SSML | Speech Synthesis Markup Language — XML-based markup for controlling TTS prosody, rate, pitch, and pausing |
| MCP | Model Context Protocol — open standard for connecting AI models to external tools and data sources without custom integration code |
| HMAC | Hash-based Message Authentication Code — cryptographic mechanism for signing and verifying webhook payloads |
| TTFB | Time To First Byte — latency from TTS synthesis start to first audio byte ready for playback |
| Anchor | Tunk.ai's chat-based agent orchestration interface — allows users to design, configure, and deploy voice AI agents in under 5 minutes through natural conversation |