API Reference - Agentic Avatars SDK

API Reference

Complete reference for all Kaltura Avatar Client SDK methods, properties, and events.

Constructor

`new KalturaAvatarSession(config?)`

Creates a new avatar session instance.

Signature:

new KalturaAvatarSession(config?: AvatarConfig)
// Legacy (deprecated): new KalturaAvatarSession(apiKey?: string, config?: AvatarConfig)

Configuration Options:

interface AvatarConfig {
  apiKey?: string; // Your Kaltura Session (KS). Required for FE-only flow; omit for BE/FE split.
  baseUrl?: string; // Avatar API root URL (e.g., https://api.avatar.us.kaltura.ai/). Defaults to https://api.avatar.us.kaltura.ai/.
  catalogUrl?: string; // Base URL for Catalog Service. Defaults to baseUrl if not provided.
  retryConfig?: RetryConfig; // Retry configuration
  logLevel?: 'debug' | 'info' | 'warn' | 'error'; // Default: 'info'
}

interface RetryConfig {
  maxAttempts: number; // Max retry attempts
  initialDelayMs: number; // Initial delay in ms
  maxDelayMs: number; // Max delay in ms
  backoffMultiplier: number; // Exponential backoff multiplier
}

Recommended Usage

Pass configuration as a single object for cleaner API:

const session = new KalturaAvatarSession({
  apiKey: 'your-kaltura-session', // optional, for FE-only flow
  baseUrl: 'https://api.avatar.us.kaltura.ai/',
  logLevel: 'debug',
});

Deprecation Notice

Passing apiKey as the first parameter is deprecated:

// Deprecated (still works, but shows warning)
const session = new KalturaAvatarSession('your-api-key', { baseUrl: '...' });

// Recommended
const session = new KalturaAvatarSession({ apiKey: 'your-api-key', baseUrl: '...' });

Examples:

// FE-Only: SDK creates the session (with apiKey in config)
const session = new KalturaAvatarSession({
  apiKey: 'your-kaltura-session',
  baseUrl: 'https://api.avatar.us.kaltura.ai/',
  logLevel: 'info',
  retryConfig: {
    maxAttempts: 3,
    initialDelayMs: 500,
    maxDelayMs: 5000,
    backoffMultiplier: 2,
  },
});

// Backend-Created: Backend creates session, FE displays (no apiKey)
const session = new KalturaAvatarSession({
  baseUrl: 'https://api.avatar.us.kaltura.ai/',
  logLevel: 'info',
});
// Then use initSession() with backend-provided credentials

Session Methods

`createSession(options)`

Create a new avatar session and establish WebRTC connection.

Parameters:

interface CreateSessionOptions {
  visualId?: string; // ID of the visual to use
  voiceId?: string; // Optional - ID of the voice for TTS
  language?: TTSLanguage; // Optional - TTS language code (e.g., 'en', 'es', 'ja')
  videoContainerId?: string; // Optional - ID of div to auto-attach video
  /** @deprecated Use `visualId` instead. Will be removed in a future major version. */
  avatarId?: string;
}

Parameter	Type	Required	Description
`visualId`	string	Yes	ID of the visual to use
`voiceId`	string	No	Voice ID for TTS
`language`	TTSLanguage	No	TTS language code (e.g., `'en'`, `'es'`)
`videoContainerId`	string	No	DOM element ID for auto-attaching video

Deprecation Notice

avatarId is deprecated in favor of visualId. It still works but will log a console warning. It will be removed in a future major version.

Returns: Promise<void>

Example:

// With auto-attach
await session.createSession({
  visualId: 'visual-123',
  voiceId: 'voice-456',
  language: 'en',
  videoContainerId: 'avatar-container',
});

// Without auto-attach
await session.createSession({
  visualId: 'visual-123',
  voiceId: 'voice-456',
});

What it does:

Initializes the backend client
Retrieves WHEP URL and TURN server configuration
Establishes WebRTC connection
Starts automatic keep-alive (every 10 seconds)
Attaches video if videoContainerId is provided

`initSession(session, options?)`

Initialize a pre-created session from your backend. Use this when your backend creates the session (recommended for production).

Parameters:

interface InitSessionParams {
  sessionId: string; // Required - from backend's create response
  token: string; // Required - JWT token from backend
}

interface InitSessionOptions {
  videoContainerId?: string; // Optional - ID of div to auto-attach video
}

Returns: Promise<void>

Example:

// Get session credentials from your backend
const { sessionId, token } = await fetch('/api/create-avatar-session').then(
  (r) => r.json()
);

// Create SDK instance WITHOUT apiKey (backend already created session)
const session = new KalturaAvatarSession({
  baseUrl: 'https://api.avatar.us.kaltura.ai/',
});

// Initialize with backend-provided credentials
await session.initSession(
  { sessionId, token },
  { videoContainerId: 'avatar-container' }
);

// SDK is now ready to use
await session.sayText('Hello from backend-created session!');

When to use:

Your backend creates the session and provides sessionId and token
For production applications (keeps Kaltura Session secure on server)
When you need server-side control and validation

What it does:

Injects the JWT token from backend
Retrieves WHEP URL and TURN server configuration
Establishes WebRTC connection
Starts automatic keep-alive (every 10 seconds)
Attaches video if videoContainerId is provided

Comparison with createSession():

Aspect	createSession	initSession
Who creates session	SDK (using apiKey)	Backend server
apiKey needed	Yes	No
Use case	Demos, prototypes	Production apps
Security	KS in client	KS on server

`attachAvatar(containerId)`

Attach avatar video to a container div by creating a video element inside it.

Parameters:

Parameter	Type	Required	Description
`containerId`	`string`	Yes	ID of the div element

Returns: void

Example:

session.attachAvatar('avatar-container');

HTML:

<div id="avatar-container" style="width: 512px; height: 512px;"></div>

Generated video element:

autoplay and playsinline attributes
Width and height set to 100%
Appropriate styling for fullscreen display

`sayText(text, turnId?, isFinal?)`

Send text for the avatar to speak using text-to-speech.

Parameters:

Parameter	Type	Required	Description
`text`	`string`	Yes	Text for the avatar to speak
`turnId`	`string`	Optional	Unique identifier for this speech turn. Auto-generated if not provided.
`isFinal`	`boolean`	Optional	Whether this is the final chunk (default: true). Use false for LLM streaming.

Returns: Promise<void>

Examples:

// Simple usage (auto-generates turnId)
await session.sayText('Hello, how are you doing today?');

// With explicit turn tracking
const turnId = `turn-${Date.now()}`;
await session.sayText('Hello!', turnId, true);

// LLM streaming support (streaming chunks from LLM)
const streamTurnId = `turn-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
await session.sayText('Hello, ', streamTurnId, false); // More chunks coming
await session.sayText('how are you?', streamTurnId, true); // Final chunk

Parameters explained:

turnId: Unique identifier to track which speech turn this text belongs to. Useful for correlating avatar speech with your application state. Use the same turnId for all chunks of a single turn.
isFinal: For simple use cases, always use true. For LLM streaming where text arrives in chunks, use false for intermediate chunks and true for the final one.

`sayAudio(audioFile, turnId, duration)`

Send an audio file for the avatar to speak.

Parameters:

Parameter	Type	Required	Description
`audioFile`	`File \| Blob`	Yes	MP3 file at 44.1 kHz
`turnId`	`string`	Yes	Unique identifier for this speech turn
`duration`	`number`	Yes	Duration of the audio in seconds

Returns: Promise<void>

Example:

// Get duration via WebAudio API
async function getAudioDuration(file: File): Promise<number> {
  const arrayBuffer = await file.arrayBuffer();
  const audioCtx = new AudioContext();
  const decoded = await audioCtx.decodeAudioData(arrayBuffer);
  await audioCtx.close();
  return decoded.duration;
}

const audioFile = new File([audioBlob], 'speech.mp3', { type: 'audio/mpeg' });
const duration = await getAudioDuration(audioFile);
const turnId = `turn-${Date.now()}`;
await session.sayAudio(audioFile, turnId, duration);

`interrupt()`

Interrupt the avatar's current speech.

Parameters: None

Returns: Promise<void>

Example:

await session.interrupt();

`endSession()`

End the session and cleanup resources.

Parameters: None

Returns: Promise<void>

Example:

await session.endSession();

What it does:

Stops the automatic keep-alive interval
Disconnects the WebRTC connection
Ends the session on the backend
Cleans up all resources

State Methods

`getSessionId()`

Get the current session ID.

Returns: string | null

Example:

const sessionId = session.getSessionId();
console.log('Session ID:', sessionId);

`getSessionState()`

Get current session state.

Returns: SessionState

Possible values:

'IDLE' - Not yet created
'CREATING' - Session being created
'READY' - Session ready for use
'ENDED' - Session ended
'ERROR' - Error occurred

Example:

const state = session.getSessionState();
console.log('Session state:', state);

`getConnectionState()`

Get current WebRTC connection state.

Returns: ConnectionState

Possible values:

'DISCONNECTED' - Not connected
'CONNECTING' - Connection in progress
'CONNECTED' - Connected and streaming
'FAILED' - Connection failed
'CLOSED' - Connection closed

Example:

const connState = session.getConnectionState();
console.log('Connection state:', connState);

Event Methods

`on(event, callback)`

Parameters:

Parameter	Type	Required	Description
`event`	`SessionEvent`	Yes	Event name
`callback`	`Function`	Yes	Event handler function

Returns: void

Available Events:

'stateChange' - Session state changed
'connectionChange' - WebRTC connection state changed
'error' - Error occurred
'speakingStart' - Avatar started speaking
'speakingEnd' - Avatar stopped speaking

`'stateChange'`

Fired when session state changes.

session.on('stateChange', (state: SessionState) => {
  console.log('Session state:', state);
});

`'connectionChange'`

Fired when WebRTC connection state changes.

session.on('connectionChange', (state: ConnectionState) => {
  console.log('Connection state:', state);
});

`'error'`

Fired when an error occurs.

session.on('error', (error: AvatarError) => {
  console.error('Error:', error.message, error.code);
});

`'speakingStart'`

Fired when the avatar starts speaking.

session.on('speakingStart', () => {
  console.log('Avatar started speaking');
  // Update UI to show speaking state
});

`'speakingEnd'`

Fired when the avatar stops speaking.

session.on('speakingEnd', () => {
  console.log('Avatar stopped speaking');
  // Update UI to show idle state
});

TypeScript Types

TTS Languages

import { TTSLanguages, TTSLanguage } from '@unisphere/models-sdk-js';

// TTSLanguages is a constant object with all supported language codes:
// ENGLISH: 'en', JAPANESE: 'ja', CHINESE: 'zh', GERMAN: 'de', HINDI: 'hi',
// FRENCH: 'fr', KOREAN: 'ko', PORTUGUESE: 'pt', ITALIAN: 'it', SPANISH: 'es',
// INDONESIAN: 'id', DUTCH: 'nl', TURKISH: 'tr', FILIPINO: 'fil', POLISH: 'pl',
// SWEDISH: 'sv', BULGARIAN: 'bg', ROMANIAN: 'ro', ARABIC: 'ar', CZECH: 'cs',
// GREEK: 'el', FINNISH: 'fi', CROATIAN: 'hr', MALAY: 'ms', SLOVAK: 'sk',
// DANISH: 'da', TAMIL: 'ta', UKRAINIAN: 'uk', RUSSIAN: 'ru', HUNGARIAN: 'hu',
// NORWEGIAN: 'no', VIETNAMESE: 'vi'

// TTSLanguage is the union type of all language code values
type TTSLanguage = 'en' | 'ja' | 'zh' | 'de' | ... ; // all 33 codes

Enums

enum SessionState {
  IDLE = 'IDLE',
  CREATING = 'CREATING',
  READY = 'READY',
  ENDED = 'ENDED',
  ERROR = 'ERROR',
}

enum ConnectionState {
  DISCONNECTED = 'DISCONNECTED',
  CONNECTING = 'CONNECTING',
  CONNECTED = 'CONNECTED',
  FAILED = 'FAILED',
  CLOSED = 'CLOSED',
}

enum AvatarErrorCode {
  INVALID_STATE = 'INVALID_STATE',
  API_REQUEST_FAILED = 'API_REQUEST_FAILED',
  RTC_CONNECTION_FAILED = 'RTC_CONNECTION_FAILED',
  INVALID_CONFIGURATION = 'INVALID_CONFIGURATION',
  SESSION_NOT_FOUND = 'SESSION_NOT_FOUND',
  UNAUTHORIZED = 'UNAUTHORIZED',
}

Error Interface

class AvatarError extends Error {
  code: AvatarErrorCode;
  message: string;
  details?: any;
}

Complete Examples

Pattern 1: FE-Only (SDK Creates Session)

import { KalturaAvatarSession, SessionState, ConnectionState, AvatarError, AvatarErrorCode } from '@unisphere/models-sdk-js';

async function runAvatarFEOnly() {
  // Create instance with apiKey in config
  const session = new KalturaAvatarSession({
    apiKey: 'your-kaltura-session',
    baseUrl: 'https://api.avatar.us.kaltura.ai/',
    logLevel: 'info',
  });

  // Event listeners
  session.on('stateChange', (state: SessionState) => {
    console.log('State:', state);
  });

  session.on('error', (error: AvatarError) => {
    console.error('Error:', error.code, error.message);
  });

  try {
    // Create session (SDK creates it using apiKey)
    await session.createSession({
      visualId: 'visual-123',
      voiceId: 'voice-456',
      language: 'en',
      videoContainerId: 'avatar-container',
    });

    console.log('Session ID:', session.getSessionId());

    // Say text
    await session.sayText('Hello from Kaltura Avatar!');

    // Wait...
    await new Promise((r) => setTimeout(r, 5000));

    // End session
    await session.endSession();
  } catch (error) {
    console.error('Failed:', error);
  }
}

Pattern 2: Backend-Created (Recommended for Production)

import { KalturaAvatarSession, SessionState, AvatarError } from '@unisphere/models-sdk-js';

async function runAvatarBECreated() {
  // Get session from your backend
  const { sessionId, token } = await fetch('/api/create-avatar-session').then(
    (r) => r.json()
  );

  // Create instance WITHOUT apiKey (no need, backend created session)
  const session = new KalturaAvatarSession({
    baseUrl: 'https://api.avatar.us.kaltura.ai/',
    logLevel: 'info',
  });

  // Event listeners
  session.on('stateChange', (state: SessionState) => {
    console.log('State:', state);
  });

  session.on('error', (error: AvatarError) => {
    console.error('Error:', error.code, error.message);
  });

  try {
    // Initialize with backend-provided credentials
    await session.initSession(
      { sessionId, token },
      { videoContainerId: 'avatar-container' }
    );

    console.log('Session ID:', session.getSessionId());

    // Say text
    const turnId = `turn-${Date.now()}`;
    await session.sayText('Hello from backend-created session!', turnId, true);

    // Wait...
    await new Promise((r) => setTimeout(r, 5000));

    // End session
    await session.endSession();
  } catch (error) {
    console.error('Failed:', error);
  }
}

Next Steps

Catalog Service - Browse available visuals and voices
State Management - Learn about states and events
Error Handling - Handle errors gracefully
Advanced Usage - Custom ICE servers and more

API Reference​

Constructor​

new KalturaAvatarSession(config?)​

Session Methods​

createSession(options)​

initSession(session, options?)​

attachAvatar(containerId)​

sayText(text, turnId?, isFinal?)​

sayAudio(audioFile, turnId, duration)​

interrupt()​

endSession()​

State Methods​

getSessionId()​

getSessionState()​

getConnectionState()​

Event Methods​

on(event, callback)​

'stateChange'​

'connectionChange'​

'error'​

'speakingStart'​

'speakingEnd'​

TypeScript Types​

TTS Languages​

Enums​

Error Interface​

Complete Examples​

Pattern 1: FE-Only (SDK Creates Session)​

Pattern 2: Backend-Created (Recommended for Production)​

Next Steps​

API Reference

Constructor

`new KalturaAvatarSession(config?)`

Session Methods

`createSession(options)`

`initSession(session, options?)`

`attachAvatar(containerId)`

`sayText(text, turnId?, isFinal?)`

`sayAudio(audioFile, turnId, duration)`

`interrupt()`

`endSession()`

State Methods

`getSessionId()`

`getSessionState()`

`getConnectionState()`

Event Methods

`on(event, callback)`

`'stateChange'`

`'connectionChange'`

`'error'`

`'speakingStart'`

`'speakingEnd'`

TypeScript Types

TTS Languages

Enums

Error Interface

Complete Examples

Pattern 1: FE-Only (SDK Creates Session)

Pattern 2: Backend-Created (Recommended for Production)

Next Steps