> ## Documentation Index > Fetch the complete documentation index at: https://docs.cloudglue.dev/llms.txt > Use this file to discover all available pages before exploring further. # Create Bulk Import > Create a bulk import: a saved definition that lists a data connector's source files and brings each match into the collection. What a run ingests follows the collection's type (see `import_type` on the response). For a **metadata collection** it imports each file's source metadata as a collection file, with no media download or processing, and runs consume no credits. For **any other collection type** it imports the media itself: each matching file is ingested and processed exactly like a manual add, so it is **billed per file** and counts against the account's file usage limits. A media run that exhausts credits or a usage limit stops with a user-facing error and keeps everything it already imported — rerun in `append` mode to resume. By default the first run starts immediately (`start: false` saves the definition only). The response's `latest_run` is the triggered run; it is null when `start` is false or when another run is already active in the collection (the definition is saved, and can be run once that one finishes), and it is returned with status `failed` when the run could not be started (trigger a new run to retry). Runs page the connector — and, for media imports, add each file — with the account's default active API key, and fail with a clear error when the account has none. ## OpenAPI ````yaml POST /collections/{collection_id}/imports openapi: 3.0.0 info: title: Cloudglue API description: API for Cloudglue license: name: Apache License 2.0 url: https://www.apache.org/licenses/LICENSE-2.0 version: 0.7.31 servers: - url: https://api.cloudglue.dev/v1 security: - bearerAuth: [] paths: /collections/{collection_id}/imports: post: tags: - Bulk Imports summary: Create a bulk import description: >- Create a bulk import: a saved definition that lists a data connector's source files and brings each match into the collection. What a run ingests follows the collection's type (see `import_type` on the response). For a **metadata collection** it imports each file's source metadata as a collection file, with no media download or processing, and runs consume no credits. For **any other collection type** it imports the media itself: each matching file is ingested and processed exactly like a manual add, so it is **billed per file** and counts against the account's file usage limits. A media run that exhausts credits or a usage limit stops with a user-facing error and keeps everything it already imported — rerun in `append` mode to resume. By default the first run starts immediately (`start: false` saves the definition only). The response's `latest_run` is the triggered run; it is null when `start` is false or when another run is already active in the collection (the definition is saved, and can be run once that one finishes), and it is returned with status `failed` when the run could not be started (trigger a new run to retry). Runs page the connector — and, for media imports, add each file — with the account's default active API key, and fail with a clear error when the account has none. operationId: createMetadataImport parameters: - name: collection_id in: path required: true description: The ID of the metadata collection schema: type: string format: uuid requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateMetadataImportRequest' responses: '200': description: >- The created import, with `latest_run` reflecting the immediately-triggered first run when applicable content: application/json: schema: $ref: '#/components/schemas/MetadataImport' '400': description: >- Invalid request — the connector does not exist, the connector type does not support imports, a filter key is not supported by the connector type, or `include_thumbnails` or `enrich_metadata` was set on a media import (imported media gets real thumbnails from the processing pipeline, and a media run has no metadata-only record to enrich) content: application/json: schema: $ref: '#/components/schemas/Error' '404': description: Collection not found content: application/json: schema: $ref: '#/components/schemas/Error' '500': description: An unexpected error occurred on the server content: application/json: schema: $ref: '#/components/schemas/Error' components: schemas: CreateMetadataImportRequest: type: object required: - name - connector_id properties: name: type: string minLength: 1 maxLength: 255 description: Display name for the import connector_id: type: string format: uuid description: >- Data connector to list files from. Supported types: google-drive, dropbox, zoom, gong, recall, grain, iconik. filters: type: array items: $ref: '#/components/schemas/MetadataImportFilterSet' maxItems: 20 default: [] description: Listing passes; empty (the default) means one unfiltered pass default_mode: type: string enum: - append - refresh default: append description: >- Mode used when a run does not specify one. append imports new files and retries previously-failed ones; refresh re-imports everything the filters match. delete_missing: type: boolean default: false description: >- On refresh runs, remove files this import previously brought in that the source no longer returns. Only files imported by this import are ever removed — manually-added files and other imports are untouched. rate_limit: type: integer minimum: 1 maximum: 50 description: >- Upstream list requests per second. Defaults to a per-connector safe rate (e.g. gong 2, zoom 6, google-drive/dropbox 8) and is clamped to a per-connector ceiling (gong 3, iconik 12, zoom/grain/recall 10, google-drive/dropbox 20) sized to keep the whole account inside each provider's published rate budget. start: type: boolean default: true description: Trigger the first run immediately. false saves the definition only. max_files: type: integer minimum: 1 maximum: 1000000 description: >- Stop each run after processing this many files from the listing. A capped run stops listing at the limit and never runs the delete-missing sweep — a truncated listing is incomplete by definition, so nothing is removed based on it. Media imports additionally cap every run at 10,000 files, whatever this is set to. include_thumbnails: type: boolean default: false description: >- Copy connector poster images as default thumbnails for imported files, for sources that provide one (Grain, iconik, Google Drive, and Dropbox today). Off by default. Posters are copied in the background, so they do not slow the import itself and may appear shortly after a file is indexed. Metadata imports only — setting it on a media import is a 400, since imported media gets real thumbnails from the processing pipeline. enrich_metadata: type: boolean description: >- Backfill source-metadata fields the connector's list endpoint omits, after each index batch settles: Gong parties + Call Spotlight content (batched - enriched docs are re-embedded so the content is searchable) and Dropbox media_info duration/dimensions (per-file). No-op for other connectors. Off by default: it spends upstream API budget and, for Gong, embedding work. Metadata imports only — setting it on a media import is a 400, since a media run ingests each file in full and has no metadata-only record to enrich. default: false MetadataImport: type: object description: >- A saved bulk import: which connector to list, how to filter it, and how runs behave. What a run ingests depends on the collection — see import_type. Definitions are immutable — delete and recreate to change one. properties: object: type: string enum: - metadata_import id: type: string format: uuid collection_id: type: string format: uuid connector_id: type: string format: uuid import_type: type: string enum: - metadata - media description: >- Inferred from the collection's type at creation and fixed for the import's lifetime. metadata (metadata collections): source metadata is indexed, no media is downloaded, runs are free. media (every other collection type): each matching file is ingested and processed exactly like a manual add, billed per file. name: type: string filters: type: array items: $ref: '#/components/schemas/MetadataImportFilterSet' description: >- Listing passes; an empty array means one unfiltered pass. Overlapping passes are deduplicated on import. default_mode: type: string enum: - append - refresh description: Mode used when a run does not specify one delete_missing: type: boolean description: Default delete-missing behavior for refresh runs rate_limit: type: integer nullable: true description: >- Upstream list requests per second; null means the per-connector safe default. Clamped to a per-connector ceiling at run time. created_at: type: number description: Unix timestamp in milliseconds latest_run: allOf: - $ref: '#/components/schemas/MetadataImportRun' nullable: true description: Most recent run, or null when the import has never run max_files: type: integer nullable: true description: >- Per-run ceiling on files processed from the listing; null means unbounded include_thumbnails: type: boolean description: >- Whether runs copy connector poster images as default thumbnails for imported files (Grain and iconik today) enrich_metadata: type: boolean description: >- Backfill source-metadata fields the connector's list endpoint omits, after each index batch settles: Gong parties + Call Spotlight content (batched - enriched docs are re-embedded so the content is searchable) and Dropbox media_info duration/dimensions (per-file). No-op for other connectors. Off by default: it spends upstream API budget and, for Gong, embedding work. Metadata imports only — setting it on a media import is a 400, since a media run ingests each file in full and has no metadata-only record to enrich. Error: required: - error type: object properties: error: type: string MetadataImportFilterSet: type: object description: >- One listing pass over the connector's source — the same query params as GET /data-connectors/{id}/files. Which keys a connector honors is validated at import creation (an unsupported key is rejected with a 400, never silently dropped): google-drive supports from/to/title_search/folder_id; dropbox from/to/title_search/path/recursive; zoom and gong from/to/title_search; grain from/to/title_search/team/meeting_type; recall from/to; iconik from/to/title_search. Zoom and Gong listings default to a 6-month lookback when no `from` is given; creation pins that date into the stored filters (six months before the set's `to` when one is given, else before creation time) so every run lists the same stable window — pass an explicit `from` to control the window yourself. iconik listings automatically window by date past iconik's 10,000-result search cap, so a single unfiltered set can cover any source size; only the degenerate case of 10,000+ assets sharing one creation timestamp still requires splitting into multiple date-window sets. additionalProperties: false properties: from: type: string format: date description: Only include source files from on/after this date (YYYY-MM-DD, UTC) to: type: string format: date description: Only include source files from on/before this date (YYYY-MM-DD, UTC) folder_id: type: string description: >- Restrict listing to a folder's direct children (Google Drive only). Omit to list the entire corpus. path: type: string description: >- Restrict listing to a folder path (Dropbox only). Dropbox listing is non-recursive: an import ingests only the direct children of this path (or of the root when omitted — typically all folders, i.e. nothing). Use one filter set per folder to cover a tree. title_search: type: string description: Only include files whose title matches this search team: type: string description: Restrict to a team (Grain only) meeting_type: type: string description: Restrict to a meeting type (Grain only) recursive: type: string enum: - 'true' - 'false' description: >- Dropbox only: 'true' imports the whole subtree under `path` (or the whole account when no `path` is given) instead of direct children. MetadataImportRun: type: object description: One execution of a metadata import. properties: object: type: string enum: - metadata_import_run id: type: string format: uuid import_id: type: string format: uuid import_type: type: string enum: - metadata - media description: >- What this run ingests, inherited from the import definition: metadata indexes source metadata only (free); media ingests and processes the actual files (billed per file). mode: type: string enum: - append - refresh description: >- append imports new files and retries previously-failed ones; refresh re-imports everything the filters match. For media imports, refresh re-syncs the source metadata of files it already imported — media bytes are never re-downloaded. delete_missing: type: boolean description: >- Whether this run sweeps files this import previously brought in that the source no longer returns (refresh runs only) status: type: string enum: - pending - processing - completed - failed - cancelled progress: $ref: '#/components/schemas/MetadataImportRunProgress' error: type: string nullable: true description: User-facing failure reason when status is failed started_at: type: number nullable: true description: Unix timestamp in milliseconds completed_at: type: number nullable: true description: Unix timestamp in milliseconds created_at: type: number description: Unix timestamp in milliseconds max_files: type: integer nullable: true description: The cap this run executed with; null means unbounded include_thumbnails: type: boolean description: Whether this run copied poster thumbnails enrich_metadata: type: boolean description: >- Whether this run backfills omitted source-metadata fields (see the import-level property). MetadataImportRunProgress: type: object description: >- Best-effort progress counters for display. Run completion is decided from the actual per-file indexing state, never from these counters, so they can drift slightly under retries without affecting correctness. properties: pages_listed: type: integer description: Connector list pages fetched files_listed: type: integer description: Source files seen across all filter sets files_created: type: integer description: New Cloudglue files created files_updated: type: integer description: Existing files re-imported (refresh mode) files_skipped: type: integer description: Already-imported files left untouched (append mode) files_failed: type: integer description: Files whose metadata indexing failed files_queued: type: integer description: >- Files handed to indexing — the denominator for progress (indexed + failed settle against it) files_indexed: type: integer description: Files whose metadata finished indexing (metadata imports) files_imported: type: integer description: >- Files whose media was ingested and added to the collection (media imports) files_removed: type: integer description: Files removed by the delete-missing sweep files_enriched: type: integer description: >- Files whose source_metadata an enrichment batch backfilled (enrich_metadata runs only). securitySchemes: bearerAuth: type: http scheme: bearer ````