Custom Audio Handling

The Cognigy Click To Call SDK handles remote audio playback automatically by creating and managing an internal Audio element. For advanced use cases, set captureAudio: true in the client configuration. The SDK then emits the captureAudio event with the raw MediaStream. The SDK also emits the audioEnded event when playback stops.

Default Behavior

By default, the SDK:

Creates an internal Audio element with autoplay enabled and volume set to 1.0.
When a remote audio stream arrives through the WebRTC peer connection, the SDK attaches it to the Audio element. Playback starts automatically.
When the call ends, the SDK stops playback, releases the stream, and emits the audioEnded event. The SDK also stops playback and emits audioEnded when you call disconnect() or destroy().

No manual audio setup is required for basic call functionality.

Browser autoplay policies might block audio playback if the call doesn’t start from a user gesture, for example, a button click. Always start calls in response to user interaction. If the browser blocks playback, the SDK emits the error event. The message starts with Audio playback failed.

Accessing the Raw Audio Stream

To receive the raw MediaStream for advanced audio processing, enable the captureAudio option and subscribe to the event:

const client = await createWebRTCClient({
  endpointUrl: 'https://your-cognigy-environment.com/token',
  userId: 'user-123',
  captureAudio: true,
});

client.on('captureAudio', (stream) => {
  // The stream contains the remote audio track
  console.log('Audio stream received:', stream.getAudioTracks());
});

The captureAudio event emits in addition to the SDK’s automatic audio playback. You don’t need to handle playback yourself. The SDK still plays audio through its internal Audio element. The SDK doesn’t provide a volume control or a way to turn off its own playback.

Audio Visualization

Create a real-time audio visualizer using the AnalyserNode:

client.on('captureAudio', (stream) => {
  const audioContext = new AudioContext();
  const source = audioContext.createMediaStreamSource(stream);
  const analyser = audioContext.createAnalyser();
  analyser.fftSize = 256;

  source.connect(analyser);
  // Don't connect to audioContext.destination — the SDK already plays audio
  // Connecting again would cause double playback

  const dataArray = new Uint8Array(analyser.frequencyBinCount);

  function draw() {
    requestAnimationFrame(draw);
    analyser.getByteFrequencyData(dataArray);

    // Use dataArray values to render a visualization
    const average = dataArray.reduce((a, b) => a + b, 0) / dataArray.length;
    console.log('Audio level:', average);
  }

  draw();
});

When you use the captureAudio stream for visualization, avoid connecting it to audioContext.destination, because the SDK already handles playback. Doing so plays the audio twice.

Audio Recording

Record the remote audio stream using the MediaRecorder API:

let mediaRecorder: MediaRecorder;
const recordedChunks: Blob[] = [];

client.on('captureAudio', (stream) => {
  // Record audio — playback is handled automatically by the SDK
  mediaRecorder = new MediaRecorder(stream, { mimeType: 'audio/webm' });
  mediaRecorder.ondataavailable = (event) => {
    if (event.data.size > 0) {
      recordedChunks.push(event.data);
    }
  };
  mediaRecorder.start();
});

client.on('audioEnded', () => {
  if (mediaRecorder && mediaRecorder.state !== 'inactive') {
    mediaRecorder.stop();
  }
});

// Save recording
function saveRecording() {
  const blob = new Blob(recordedChunks, { type: 'audio/webm' });
  const url = URL.createObjectURL(blob);
  const a = document.createElement('a');
  a.href = url;
  a.download = 'call-recording.webm';
  a.click();
}

Recording calls may be subject to legal requirements such as consent laws. Ensure your application complies with applicable regulations before recording audio.

Overview

Tutorials

Click To Call SDK

Default Behavior

Accessing the Raw Audio Stream

Audio Visualization

Audio Recording

More Information

​Default Behavior

​Accessing the Raw Audio Stream

​Audio Visualization

​Audio Recording

​More Information

Default Behavior

Accessing the Raw Audio Stream

Audio Visualization

Audio Recording

More Information