Skip to main content
Intermediate · 20 minutes · Voice
Speech to speech lets a voice agent hear and speak directly, without separate speech-to-text and text-to-speech providers. In this tutorial, you turn it on for the voice agent Nelly Voice and test it with a voice call in your browser.
Speech to speech is an experimental feature. Settings and behavior can change.
Two models work together:

​
Prerequisites

Complete the following prerequisites:
  • The voice agent Nelly Voice. To create the agent, follow Getting started with voice agents.
  • An LLM that uses the model gpt-live-1. To add an LLM, go to Manage > AI & Voice Providers in your project. For details, see LLMs.
  • A browser with microphone access allowed. The test call uses your microphone.

​
Steps

1

Open the agent

  1. In the navigation menu of the project, go to Design > Agent and open Nelly Voice.
  2. Go to the Configuration tab. The LLM and Voice sections show the model and speech providers that the agent uses.
2

Turn on speech to speech

  1. Find the card labeled Experimental. The card says that the agent hears and speaks directly.
  2. Turn on the toggle in the card.
The Voice model, Delegation model, Agent voice, and Voice prompt settings replace the LLM and Voice sections.
3

Choose the models and the voice

  1. From the Voice model list, select an LLM that uses gpt-live-1, for example, GPT Live.
  2. From the Delegation model list, keep the preselected model.
  3. From the Agent voice list, select a voice, for example, Marin.
Speech to speech settings with the voice model GPT Live, the delegation model, the voice Marin, and the Voice prompt card.
4

Review the voice prompt

The voice prompt is the full prompt that the voice model follows.
  1. In the Voice prompt card, click Open voice prompt. The Voice prompt page opens.
  2. Read the Prompt template. The Prompt outline lists the template sections: Backchannel policy, Interruption policy, Delegation policy, Opening, AI Disclosure, and Language.
  3. Find the # Opening section. By default, the agent speaks first when the call connects. It introduces itself by name as an AI assistant and asks how it can help.
  4. (Optional) To change how Nelly Voice opens the call, edit the text under # Opening and click Save changes.
The Voice prompt page with the Edit and Preview switch, the Prompt outline, and the prompt template that starts with the agent name variable.
The voice prompt doesn’t include the Instructions from the Configuration tab. The Voice prompt page notes that the Instructions go to the backend model.
5

Save the agent

Go back to the Configuration tab and click Save changes.
6

Test the agent with a voice call

  1. To open the interaction panel, click the chat icon in the upper-right corner.
  2. At the top of the panel, select Voice.
  3. Click Start a call. If your browser asks for microphone access, allow it.
  4. Listen to Nelly Voice. The agent speaks first. For example, it says: “Hi, I’m Nelly Voice, an AI assistant. How can I help you today?”
  5. Ask a question, for example, “What can you help me with?”
  6. Interrupt the agent while it speaks. In speech to speech, the agent might not stop speaking right away. By default, the voice prompt tells the agent to ignore short sounds such as “mm-hm”. It listens when you start a new request.
  7. End the call in the interaction panel.
The call runs in your browser. Any action in voice mode ends the current session and starts a new one.
The interaction panel with Voice selected and the Start a call button below the agent avatar of Nelly Voice.
To switch back to speech-to-text and text-to-speech providers, turn off the toggle on the Configuration tab and click Save changes.

​
What you built

You switched Nelly Voice to speech to speech. You chose the voice model, the delegation model, and a voice, reviewed the voice prompt, and tested the agent with a voice call.

​
More information

Last modified on November 11, 2026