Chat Over Collection

Talk with videos

Quick Start

First, make sure you have Cloudglue set up by following the setup guide.

Create a Collection and Add Videos

Using Cloudglue’s Chat Completion API, you can have natural conversations about your video content. The API enables you to ask questions, get summaries, and extract insights from your video collections. When you make a chat request, Cloudglue intelligently searches through your video transcripts, scene descriptions, and detected text to provide accurate, context-aware responses with relevant citations.

To chat with your videos, you’ll need to:

1. Create a rich transcript collection
2. Add the videos to your collection
3. Start chatting!

Here’s how to do it:

Python

from cloudglue import CloudGlue

# Initialize client (API key will be loaded from environment)
client = CloudGlue()

# Create a rich transcript collection
collection = client.collections.create(
    name='my-video-collection',
    collection_type='rich-transcripts'
)

# Upload two video files
video1 = client.files.upload(
    'path/to/first/video.mp4',
    wait_until_finish=True
)
video2 = client.files.upload(
    'path/to/second/video.mp4',
    wait_until_finish=True
)

# Add videos to collection (waits for processing to complete)
client.collections.add_video(
    collection_id=collection.id,
    file_id=video1.id,
    wait_until_finish=True
)
client.collections.add_video(
    collection_id=collection.id,
    file_id=video2.id,
    wait_until_finish=True
)

# Chat with your videos!
messages = [
    {"role": "user", "content": "Who let the dogs out?"}
]

response = client.chat.completions.create(
    messages=messages,
    model="nimbus-001",
    collections=[collection.id],
    force_search=True,
    include_citations=True
)

print(response.choices[0].message.content)

For more details about the Python SDK, check out our Python SDK Documentation.

from cloudglue import CloudGlue

# Initialize client (API key will be loaded from environment)
client = CloudGlue()

# Create a rich transcript collection
collection = client.collections.create(
    name='my-video-collection',
    collection_type='rich-transcripts'
)

# Upload two video files
video1 = client.files.upload(
    'path/to/first/video.mp4',
    wait_until_finish=True
)
video2 = client.files.upload(
    'path/to/second/video.mp4',
    wait_until_finish=True
)

# Add videos to collection (waits for processing to complete)
client.collections.add_video(
    collection_id=collection.id,
    file_id=video1.id,
    wait_until_finish=True
)
client.collections.add_video(
    collection_id=collection.id,
    file_id=video2.id,
    wait_until_finish=True
)

# Chat with your videos!
messages = [
    {"role": "user", "content": "Who let the dogs out?"}
]

response = client.chat.completions.create(
    messages=messages,
    model="nimbus-001",
    collections=[collection.id],
    force_search=True,
    include_citations=True
)

print(response.choices[0].message.content)

For more details about the Python SDK, check out our Python SDK Documentation.

import { CloudGlue } from '@aviaryhq/cloudglue-js';
import * as fs from 'fs';
import * as path from 'path';

const client = new CloudGlue();

// Create a rich transcript collection
const collection = await client.collections.createCollection({
  name: 'my-video-collection',
  collection_type: 'rich-transcripts'
});

// Upload and process first video
const file1 = new File(
  [await fs.promises.readFile('path/to/first/video.mp4')],
  'video1.mp4'
);
const upload1 = await client.files.uploadFile({ file: file1 });
await client.files.waitForReady(upload1.id);

const add1 = await client.collections.addVideo(collection.id, upload1.data.id);
await client.collections.waitForReady(collection.id, add1.file_id);

// Upload and process second video
const file2 = new File(
  [await fs.promises.readFile('path/to/second/video.mp4')],
  'video2.mp4'
);
const upload2 = await client.files.uploadFile({ file: file2 });
await client.files.waitForReady(upload2.id);

const add2 = await client.collections.addVideo(collection.id, upload2.data.id);
await client.collections.waitForReady(collection.id, add2.file_id);

// Chat with your videos!
const chatResult = await client.chat.createCompletion({
  model: 'nimbus-001',
  messages: [
    {
      role: 'user',
      content: 'Who let the dogs out?'
    }
  ],
  collections: [collection.id],
  force_search: true,
  include_citations: true
});

console.log(chatResult.choices[0].message.content);

For more details about the Node.js SDK, check out our JavaScript SDK Documentation.

For more detailed examples and advanced usage, check out our API Reference.