Twilio Integration

Learn how to integrate a Conversational AI agent with Twilio to create seamless, human-like voice interactions.

Connect your ElevenLabs Conversational AI agent to phone calls and create human-like voice experiences using Twilio’s Voice API.

What You’ll Need

• An ElevenLabs account
• A configured ElevenLabs Conversational Agent (create one here)
• A Twilio account with an active phone number
• Python 3.7+ or Node.js 16+
• ngrok for local development

Agent Configuration

Before integrating with Twilio, you’ll need to configure your agent to use the correct audio format supported by Twilio.

1. Configure TTS Output

   1. Navigate to your agent settings
   2. Go to the Voice Section
   3. Select “μ-law 8000 Hz” from the dropdown

   
2. Set Input Format

   1. Navigate to your agent settings
   2. Go to the Advanced Section
   3. Select “μ-law 8000 Hz” for the input format

   

Implementation

Javascript

Looking for a complete example? Check out this Javascript implementation on GitHub.

1. Initialize the Project

   First, set up a new Node.js project:

   mkdir conversational-ai-twilio
   cd conversational-ai-twilio
   npm init -y; npm pkg set type="module";
   

2. Install dependencies

   Next, install the required dependencies for the project.

   npm install @fastify/formbody @fastify/websocket dotenv fastify ws
   

3. Create the project files

   Create a .env & index.js file with the following code:

   .env

   ELEVENLABS_AGENT_ID=<your-agent-id>
   

4. Run the server

   You can now run the server with the following command:

   node index.js
   

   If the server starts successfully, you should see the message [Server] Listening on port 8000 (or the port you specified) in your terminal.

Twilio Setup

1. Create a Public URL

   Use ngrok to make your local server accessible:

   ngrok http --url=<your-url-here> 8000
   

   
2. Configure Twilio

   1. Go to the Twilio Console
   2. Navigate to Phone Numbers → Manage → Active numbers
   3. Select your phone number
   4. Under “Voice Configuration”, set the webhook for incoming calls to: https://your-ngrok-url.ngrok.app/incoming-call-eleven
   5. Set the HTTP method to POST

   

Testing

1. Call your Twilio phone number.
2. Start speaking - you’ll see the transcripts in the ElevenLabs console.

Troubleshooting

Connection Issues

If the WebSocket connection fails:

• Verify your ngrok URL is correct in Twilio settings
• Check that your server is running and accessible
• Ensure your firewall isn’t blocking WebSocket connections

Audio Problems

If there’s no audio output:

• Confirm your ElevenLabs API key is valid
• Verify the AGENT_ID is correct
• Check audio format settings match Twilio’s requirements (μ-law 8kHz)

Security Best Practices

Follow these security guidelines for production deployments:

• Use environment variables for sensitive information
• Implement proper authentication for your endpoints
• Use HTTPS for all communications
• Regularly rotate API keys
• Monitor usage to prevent abuse