Choosing Your Approach - Agentic Avatars SDK

Choosing Your Approach

The Kaltura Avatar system offers two control approaches for interacting with avatars.

Quick Comparison

Feature	SDK-Based Control	API-Based Control
Complexity	Simple	Moderate
Security	KS in client	KS on server ✅
Architecture	Client-side only	Backend + Frontend
Best for	Demos, prototypes	Production apps ✅
Setup time	5 minutes	15 minutes
Control location	Browser	Server
Scalability	Limited	High ✅

SDK-Based Control

How It Works

Everything runs in the browser. The SDK handles both avatar display and control.

import { KalturaAvatarSession } from '@unisphere/models-sdk-js';

const session = new KalturaAvatarSession({
  apiKey: 'your-api-key',
  baseUrl: 'https://api.avatar.us.kaltura.ai/',
});

await session.createSession({
  visualId: 'visual-123',
  videoContainerId: 'avatar-container',
});

await session.sayText('Hello from Kaltura Avatar!');

Pros

• Simple: Only one SDK needed, no backend required
• Fast to implement: Operational within approximately 5 minutes
• Easy debugging: All activity visible in browser DevTools
• Perfect for demos: Ideal for rapid prototypes and presentations

Cons

• Security risk: The Kaltura Session is exposed in client-side code
• Limited control: No ability to validate or modify from the server
• No audit trail: Server-side action logging is unavailable
• Client dependency: Relies entirely on the client implementation

When to Use

✅ Prototypes and demos ✅ Internal tools (not publicly accessible) ✅ Learning and experimentation ✅ Client-only applications

❌ Public-facing production apps ❌ Apps requiring server-side validation ❌ Apps needing audit trails

API-Based Control (Recommended)

How It Works

The backend holds the Kaltura Session and controls the avatar via API. The frontend uses the Client SDK only for display, receiving a JWT token from the backend.

SDK apiKey Parameter

The KalturaAvatarSession constructor accepts an optional apiKey parameter:

• Pass it when using frontend-only control (e.g., new KalturaAvatarSession('your-ks'))
• Omit it for the backend/frontend split approach (e.g., new KalturaAvatarSession() or new KalturaAvatarSession(undefined, ...))

With API-based control, the backend holds the Kaltura Session while the frontend receives only a JWT token through initSession().

Backend (Node.js):

const response = await fetch(
  'https://api.avatar.us.kaltura.ai/v1/avatar-session/create',
  {
    method: 'POST',
    headers: {
      Authorization: `ks ${process.env.AVATAR_KS}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      clientId: 'kaltura-avatar-sdk',
      visualConfig: { id: 'avatar-123' },
      voiceConfig: { id: 'voice-456', modelId: 'eleven_flash_v2_5' },
    }),
  }
);

const { sessionId, token } = await response.json();
res.json({ sessionId, token });

Frontend:

import { KalturaAvatarSession } from '@unisphere/models-sdk-js';

const { sessionId, token } = await fetch('/api/create-avatar-session')
  .then((r) => r.json());

const session = new KalturaAvatarSession(undefined, {
  baseUrl: 'https://api.avatar.us.kaltura.ai/',
});

await session.initSession(
  { sessionId, token },
  { videoContainerId: 'avatar-container' }
);

Pros

• Secure: The Kaltura Session remains on the server
• Separation: Backend handles control while frontend handles display
• Validation: The server validates all actions
• Audit trail: All avatar interactions can be logged
• Scalable: The backend can manage multiple sessions simultaneously

Cons

• More complex: Backend infrastructure is required
• Additional code: Coordination between backend and frontend is necessary
• Network latency: An extra network hop through the backend is introduced

When to Use

✅ Production applications (recommended) ✅ Public-facing websites ✅ Apps requiring security ✅ Apps needing server-side logic ✅ Apps with user authentication ✅ Apps requiring audit logs

Can I Switch Later?

Yes — the Client SDK always handles display regardless of approach. The recommended migration path:

1. Start with SDK-based control for rapid development
2. Switch to API-based control when moving to production

Migration involves keeping the Client SDK for display, moving control calls from the SDK to the API, and storing the Kaltura Session on the backend.

Hybrid Approach

A combined strategy is also possible:

• The Client SDK displays the avatar
• Simple actions are controlled via the SDK
• Critical or sensitive actions go through the API for validation

This offers flexibility but introduces additional complexity.

Related Guides

• Conversational Agents — Server-side APIs for provisioning avatars, intellects, knowledge, and agents (the management layer behind avatar sessions)
• Avatar Videos — Pre-recorded avatar video generation (not real-time) via server-side API or browser widget

Next Steps

• SDK Quick Start — get started with SDK-based control
• API Quick Start — get started with API-based control
• Client SDK Reference — complete SDK documentation
• Control API Reference — complete API documentation