On-Demand Describe

Learn about on-demand describe operations

What is On-Demand Media Description?

On-Demand Media Description allows you to describe a video, exactly as you need it, when you need it.

It is a great to use when you have one-off videos that you want to describe, or, when you want to get deeper descriptions for just one or a set of particular videos.

To learn more about the different types of media descriptions that Cloudglue offers here.

You can use this to quickly get the speech transcripts of a single YouTube video, or get more in-depth descriptions that include scenes and visual descriptions for a different video.

You can control the level of description you want to perform, to get the most out of your videos.

If you’re finding that you need to describe a lot of videos, in the same format, then you should consider using a Media Description Collection.

How to use it

You can describe a video on-demand by using the Describe API.

Describing images

On-Demand Describe also works on still image files. Images are described at the file level only (no segmentation) and billed at a flat 1 credit per request. Because an image has no audio track, enable_speech and enable_audio_description are not available for images — describe defaults to a visual description (visual scene description, scene text, and summary).

Speaker identification

When enable_speech is on, the transcript labels each turn with a speaker. For files added through a data connector that carries an attendee list (e.g. Grain), those real names are used automatically.

For uploaded files — which have no attendee metadata — you can pass the cast yourself via the optional participants field. When provided, speaker naming is constrained to that list: each speaker is matched to one of the supplied names or left as a generic Speaker N, and names that aren’t on the list are never invented.

{
  "url": "cloudglue://files/your-file-id",
  "enable_speech": true,
  "participants": [
    { "name": "Alice Smith", "scope": "internal" },
    { "name": "Bob Jones", "scope": "external" }
  ]
}

Guiding the description with a prompt

Every describe request accepts an optional prompt: free-form guidance that tells the description passes what matters in this media. It is the right place for the things a model cannot know from the pixels alone — the spelling of your product names and acronyms, the jargon of your domain, who tends to be on these calls, or what you want called out.

{
  "url": "cloudglue://files/your-file-id",
  "enable_speech": true,
  "enable_scene_text": true,
  "prompt": "Enterprise sales call. Spell product names as Cloudglue, Aviary, and Foreman. The host is Alice Smith. Call out any pricing, contract terms, or renewal dates discussed."
}

The prompt is applied to the visual, scene-text, speech, and summary passes, and to the file-level title and summary. Two things it deliberately does not do:

• It cannot add content that isn’t there. Naming a term or a person is context, not an instruction to report them — if they don’t appear in the media, the description stays silent about them.
• It does not constrain speaker labels. For that, use participants, which is a closed set: a speaker is matched to one of those names or left as Speaker N. A prompt is a hint the speaker-identification pass weighs as background, never as evidence.

The prompt is part of the describe config, so it is also part of the cache key: describing the same file under different guidance produces a new job rather than reusing the earlier result. It is limited to 2000 characters.

Describing many files under the same guidance? Set describe_config.prompt once on a Media Description Collection instead of repeating it per request.

Chat Completion

If you are looking to chat with your videos, you should consider using a Media Description Collection, as chat completion is not supported for on-demand descriptions.