Create Bulk Import

Create a bulk import: a saved definition that lists a data connector’s source files and brings each match into the collection. What a run ingests follows the collection’s type (see import_type on the response). For a metadata collection it imports each file’s source metadata as a collection file, with no media download or processing, and runs consume no credits. For any other collection type it imports the media itself: each matching file is ingested and processed exactly like a manual add, so it is billed per file and counts against the account’s file usage limits. A media run that exhausts credits or a usage limit stops with a user-facing error and keeps everything it already imported — rerun in append mode to resume. By default the first run starts immediately (start: false saves the definition only). The response’s latest_run is the triggered run; it is null when start is false or when another run is already active in the collection (the definition is saved, and can be run once that one finishes), and it is returned with status failed when the run could not be started (trigger a new run to retry). Runs page the connector — and, for media imports, add each file — with the account’s default active API key, and fail with a clear error when the account has none.

POST /collections/{collection_id}/imports

Create a bulk import

cURL

curl --request POST \
  --url https://api.cloudglue.dev/v1/collections/{collection_id}/imports \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "<string>",
  "connector_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "filters": [],
  "default_mode": "append",
  "delete_missing": false,
  "rate_limit": 25,
  "start": true,
  "max_files": 500000,
  "include_thumbnails": false,
  "enrich_metadata": false
}
'

200

{
  "object": "metadata_import",
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "collection_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "connector_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "import_type": "metadata",
  "name": "<string>",
  "filters": [
    {
      "from": "2023-12-25",
      "to": "2023-12-25",
      "folder_id": "<string>",
      "path": "<string>",
      "title_search": "<string>",
      "team": "<string>",
      "meeting_type": "<string>",
      "recursive": "true"
    }
  ],
  "default_mode": "append",
  "delete_missing": true,
  "rate_limit": 123,
  "created_at": 123,
  "latest_run": {
    "object": "metadata_import_run",
    "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "import_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "import_type": "metadata",
    "mode": "append",
    "delete_missing": true,
    "status": "pending",
    "progress": {
      "pages_listed": 123,
      "files_listed": 123,
      "files_created": 123,
      "files_updated": 123,
      "files_skipped": 123,
      "files_failed": 123,
      "files_queued": 123,
      "files_indexed": 123,
      "files_imported": 123,
      "files_removed": 123,
      "files_enriched": 123
    },
    "error": "<string>",
    "started_at": 123,
    "completed_at": 123,
    "created_at": 123,
    "max_files": 123,
    "include_thumbnails": true,
    "enrich_metadata": true
  },
  "max_files": 123,
  "include_thumbnails": true,
  "enrich_metadata": true
}

Authorizations

Authorization · string · header · required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

collection_id · string<uuid> · required

The ID of the metadata collection

Body

application/json

name · string · required

Display name for the import

Required string length: 1 - 255

connector_id · string<uuid> · required

Data connector to list files from. Supported types: google-drive, dropbox, zoom, gong, recall, grain, iconik.

filters · object[]

Listing passes; empty (the default) means one unfiltered pass

Maximum array length: 20

Show child attributes

default_mode · enum<string> · default: append

Mode used when a run does not specify one. append imports new files and retries previously-failed ones; refresh re-imports everything the filters match.

Available options: append, refresh

delete_missing · boolean · default: false

On refresh runs, remove files this import previously brought in that the source no longer returns. Only files imported by this import are ever removed — manually-added files and other imports are untouched.

rate_limit · integer

Upstream list requests per second. Defaults to a per-connector safe rate (e.g. gong 2, zoom 6, google-drive/dropbox 8) and is clamped to a per-connector ceiling (gong 3, iconik 12, zoom/grain/recall 10, google-drive/dropbox 20) sized to keep the whole account inside each provider's published rate budget.

Required range: 1 <= x <= 50

start · boolean · default: true

Trigger the first run immediately. false saves the definition only.

max_files · integer

Stop each run after processing this many files from the listing. A capped run stops listing at the limit and never runs the delete-missing sweep — a truncated listing is incomplete by definition, so nothing is removed based on it. Media imports additionally cap every run at 10,000 files, whatever this is set to.

Required range: 1 <= x <= 1000000

include_thumbnails · boolean · default: false

Copy connector poster images as default thumbnails for imported files, for sources that provide one (Grain, iconik, Google Drive, and Dropbox today). Off by default. Posters are copied in the background, so they do not slow the import itself and may appear shortly after a file is indexed. Metadata imports only — setting it on a media import is a 400, since imported media gets real thumbnails from the processing pipeline.

enrich_metadata · boolean · default: false

Backfill source-metadata fields the connector's list endpoint omits, after each index batch settles: Gong parties + Call Spotlight content (batched - enriched docs are re-embedded so the content is searchable) and Dropbox media_info duration/dimensions (per-file). No-op for other connectors. Off by default: it spends upstream API budget and, for Gong, embedding work. Metadata imports only — setting it on a media import is a 400, since a media run ingests each file in full and has no metadata-only record to enrich.

Response

200

application/json

The created import, with latest_run reflecting the immediately-triggered first run when applicable

A saved bulk import: which connector to list, how to filter it, and how runs behave. What a run ingests depends on the collection — see import_type. Definitions are immutable — delete and recreate to change one.

object · enum<string>

Available options: metadata_import

id · string<uuid>

collection_id · string<uuid>

connector_id · string<uuid>

import_type · enum<string>

Inferred from the collection's type at creation and fixed for the import's lifetime. metadata (metadata collections): source metadata is indexed, no media is downloaded, runs are free. media (every other collection type): each matching file is ingested and processed exactly like a manual add, billed per file.

Available options: metadata, media

name · string

filters · object[]

Listing passes; an empty array means one unfiltered pass. Overlapping passes are deduplicated on import.

Show child attributes

default_mode · enum<string>

Mode used when a run does not specify one

Available options: append, refresh

delete_missing · boolean

Default delete-missing behavior for refresh runs

rate_limit · integer | null

Upstream list requests per second; null means the per-connector safe default. Clamped to a per-connector ceiling at run time.

created_at · number

Unix timestamp in milliseconds

latest_run · object | null

Most recent run, or null when the import has never run

Show child attributes

max_files · integer | null

Per-run ceiling on files processed from the listing; null means unbounded

include_thumbnails · boolean

Whether runs copy connector poster images as default thumbnails for imported files (Grain and iconik today)

enrich_metadata · boolean

Backfill source-metadata fields the connector's list endpoint omits, after each index batch settles: Gong parties + Call Spotlight content (batched - enriched docs are re-embedded so the content is searchable) and Dropbox media_info duration/dimensions (per-file). No-op for other connectors. Off by default: it spends upstream API budget and, for Gong, embedding work. Metadata imports only — setting it on a media import is a 400, since a media run ingests each file in full and has no metadata-only record to enrich.