Skip to main content
View as Markdown

Overview - Agentic Avatars SDK

Control API Overview​

The Kaltura Avatar Control API allows you to control avatars from your backend server using REST API calls.

Recommended Approach

For production applications, use the Control API for server-to-server communication. This keeps your API key secure and provides better architectural separation.

Why Use the Control API?​

Security​

  • API key stays on server - Never exposed to client code
  • Server-side validation - Validate all actions before execution
  • Token-based access - JWTs for frontend, API key for backend

Architecture​

  • Separation of concerns - Backend controls, frontend displays
  • Audit trail - Log all avatar interactions server-side
  • Business logic - Add validation, rate limiting, user management

Scalability​

  • Centralized control - Manage multiple sessions from one place
  • Resource management - Better control over session lifecycle
  • Load balancing - Distribute load across backend servers

Architecture Pattern​

┌─────────────┐      ┌─────────────┐      ┌──────────────┐
│   Browser   │◄────►│   Backend   │◄────►│ Avatar API   │
│             │      │             │      │              │
│ Client SDK  │      │ Your Server │      │ (Kaltura)    │
│ (Display)   │      │ (Control)   │      │              │
└─────────────┘      └─────────────┘      └──────────────┘
 JWT token            API key              Sessions
 Display only         Full control         Authentication

Flow​

  1. Frontend requests session from your backend
  2. Backend calls Kaltura API with API key
  3. Backend receives sessionId + JWT token
  4. Backend returns sessionId + token to frontend
  5. Frontend uses Client SDK for display (with token)
  6. Backend controls avatar via API calls

API Base URL​

https://api.avatar.us.kaltura.ai/v1/avatar-session

Authentication​

The API uses two authentication methods:

1. Kaltura Session (Authorization: KS header)​

Used for creating sessions.

curl -X POST https://api.avatar.us.kaltura.ai/v1/avatar-session/create \
  -H "Authorization: KS your-kaltura-session" \
  -H "Content-Type: application/json" \
  -d '{"visualConfig": {"id": "visual-123"}}'

2. JWT Token (Bearer token)​

Used for all other operations.

curl -X POST https://api.avatar.us.kaltura.ai/v1/avatar-session/:sessionId/say-text \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello!"}'

See Authentication for details.

Available Endpoints​

MethodEndpointDescriptionAuth
POST/v1/avatar-session/createCreate new sessionKS
POST/v1/avatar-session/:sessionId/init-clientInitialize clientBearer token
POST/v1/avatar-session/:sessionId/say-textMake avatar speak textBearer token
POST/v1/avatar-session/:sessionId/say-audioMake avatar speak audioBearer token
POST/v1/avatar-session/:sessionId/interruptInterrupt avatarBearer token
POST/v1/avatar-session/:sessionId/keep-aliveKeep session aliveBearer token
POST/v1/avatar-session/:sessionId/endEnd sessionBearer token

Quick Example​

Backend (Node.js)​

import express from 'express';
import fetch from 'node-fetch';

const app = express();
const AVATAR_KS = process.env.AVATAR_KS;
const AVATAR_BASE_URL = 'https://api.avatar.us.kaltura.ai/v1/avatar-session';

// Create session
app.post('/api/avatar/create', async (req, res) => {
  const response = await fetch(`$\{AVATAR_BASE_URL\}/create`, {
    method: 'POST',
    headers: {
      Authorization: `KS $\{AVATAR_KS\}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      visualConfig: { id: 'visual-123' },
      voiceConfig: { id: 'voice-456' },
    }),
  });

  const data = await response.json();
  res.json({
    sessionId: data.sessionId,
    token: data.token,
  });
});

// Make avatar speak
app.post('/api/avatar/speak', async (req, res) => {
  const { sessionId, token, text } = req.body;

  await fetch(`$\{AVATAR_BASE_URL\}/${sessionId}/say-text`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${token}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ text }),
  });

  res.json({ success: true });
});

Frontend​

// Get session from your backend
const { sessionId, token } = await fetch('/api/avatar/create').then((r) => r.json());

// Use Client SDK for display
import { KalturaAvatarSession } from '@unisphere/models-sdk-js';

const session = new KalturaAvatarSession(token, {
  baseUrl: 'https://api.avatar.us.kaltura.ai/',
});

// Display avatar
session.attachAvatar('avatar-container');

// Control from backend
await fetch('/api/avatar/speak', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    sessionId,
    token,
    text: 'Hello from the backend!',
  }),
});

Session Lifecycle​

Create Session → Init Client → Use Avatar → Keep Alive → End Session
     (KS)        (Token)      (Token)      (Token)       (Token)
  1. Create Session - Get sessionId + token
  2. Init Client - Get WebRTC config (WHEP URL, TURN)
  3. Use Avatar - Call say-text, say-audio, interrupt
  4. Keep Alive - Send every 10 seconds (or use SDK auto-keep-alive)
  5. End Session - Clean up resources
Auto Keep-Alive

If using the Client SDK, keep-alive is automatic. If not, you must call /keep-alive every 10 seconds.

Error Handling​

All endpoints return standard error responses:

{
  "success": false,
  "error": "Error message here"
}

Common HTTP status codes:

  • 200 - Success
  • 400 - Bad request (invalid parameters)
  • 401 - Unauthorized (invalid API key or token)
  • 404 - Not found (session doesn't exist)
  • 410 - Gone (session ended or expired)
  • 500 - Server error

Rate Limits​

Contact your Kaltura representative for rate limit information.

Next Steps​