Also see the Conversational AI overview

​
Installation

Add the ElevenLabs Swift SDK to your project using Swift Package Manager:
1

Add the Package Dependency

  1. Open your project in Xcode
  2. Go to File > Add Packages...
  3. Enter the repository URL: https://github.com/elevenlabs/ElevenLabsSwift
  4. Select your desired version
2

Import the SDK

import ElevenLabsSDK
Ensure you add NSMicrophoneUsageDescription to your Info.plist to explain microphone access to users.

​
Usage

This library is primarily designed for Conversational AI integration in Swift applications. Please use an alternative dependency for other features, such as speech synthesis.

​
Initialize Conversation

First, create a session configuration and set up the necessary callbacks: 
// Configure the session
let config = ElevenLabsSDK.SessionConfig(agentId: "your-agent-id")

// Set up callbacks
var callbacks = ElevenLabsSDK.Callbacks()
callbacks.onConnect = { conversationId in
    print("Connected with ID: \(conversationId)")
}
callbacks.onDisconnect = {
    print("Disconnected")
}
callbacks.onMessage = { message, role in
    print("\(role.rawValue): \(message)")
}
callbacks.onError = { error, info in
    print("Error: \(error), Info: \(String(describing: info))")
}
callbacks.onStatusChange = { status in
    print("Status changed to: \(status.rawValue)")
}
callbacks.onModeChange = { mode in
    print("Mode changed to: \(mode.rawValue)")
}
callbacks.onVolumeUpdate = { volume in
    print("Volume updated: \(volume)")
}

​
Session Configuration

There are two ways to initialize a session:
You can obtain an Agent ID through the ElevenLabs UI:
let config = ElevenLabsSDK.SessionConfig(agentId: "<your-agent-id>")

​
Client Tools

Client Tools allow you to register custom functions that can be called by your AI agent during conversations. This enables your agent to perform actions in your application.

​
Registering Tools

Register custom tools before starting a conversation: 
// Create client tools instance
var clientTools = ElevenLabsSDK.ClientTools()

// Register a custom tool with an async handler
clientTools.register("generate_joke") { parameters async throws -> String? in
    // Parameters is a [String: Any] dictionary
    guard let joke = parameters["joke"] as? String else {
        throw ElevenLabsSDK.ClientToolError.invalidParameters
    }
    print("generate_joke tool received joke: \(joke)")

    return joke
}
Remember to setup your agent with the client-tools in the ElevenLabs UI. See the Client Tools documentation for setup instructions.

​
Starting the Conversation

Initialize the conversation session asynchronously: 
Task {
    do {
        let conversation = try await ElevenLabsSDK.Conversation.startSession(
            config: config,
            callbacks: callbacks,
            clientTools: clientTools // Optional: pass the previously configured client tools
        )
        // Use the conversation instance
    } catch {
        print("Failed to start conversation: \(error)")
    }
}
The client tools parameter is optional. If you don’t need custom tools, you can omit it when starting the session.

​
Audio Sample Rates

The ElevenLabs SDK currently uses a default input sample rate of 16,000 Hz. However, the output sample rate is configurable based on the agent’s settings. Ensure that the output sample rate aligns with your specific application’s audio requirements for smooth interaction.
The SDK does not currently support ulaw format for audio encoding. For compatibility, consider using alternative formats.

​
Managing the Session

// Starts the session
conversation.startSession()
// Ends the session
conversation.endSession()

​
Example Implementation

For a full, working example, check out the example application on GitHub. Here’s an example SwiftUI view implementing the conversation interface: 
struct ConversationalAIView: View {
    @State private var conversation: ElevenLabsSDK.Conversation?
    @State private var mode: ElevenLabsSDK.Mode = .listening
    @State private var status: ElevenLabsSDK.Status = .disconnected
    @State private var audioLevel: Float = 0.0

    private func startConversation() {
        Task {
            do {
                let config = ElevenLabsSDK.SessionConfig(agentId: "your-agent-id")
                var callbacks = ElevenLabsSDK.Callbacks()

                callbacks.onConnect = { conversationId in
                    status = .connected
                }
                callbacks.onDisconnect = {
                    status = .disconnected
                }
                callbacks.onModeChange = { newMode in
                    DispatchQueue.main.async {
                        mode = newMode
                    }
                }
                callbacks.onVolumeUpdate = { newVolume in
                    DispatchQueue.main.async {
                        audioLevel = newVolume
                    }
                }

                conversation = try await ElevenLabsSDK.Conversation.startSession(
                    config: config,
                    callbacks: callbacks
                )
            } catch {
                print("Failed to start conversation: \(error)")
            }
        }
    }

    var body: some View {
        VStack {
            // Your UI implementation
            Button(action: startConversation) {
                Text(status == .connected ? "End Call" : "Start Call")
            }
        }
    }
}
This SDK is currently experimental and under active development. While it’s stable enough for testing and development, it’s not recommended for production use yet.