Cloud
Also in Self-Managed

google_drive_search

Searches Google Drive for files that match a specified query and emits the results as a batch of messages. Each message contains the metadata of a Google Drive file.

Try out the example pipeline on this page, which searches for and downloads all Google Drive files that match the specified query.

  • Common

  • Advanced

processor:
  label: ""
  google_drive_search:
    credentials_json: "" # No default (optional)
    query: "" # No default (required)
    projection:
      - "id"
      - "name"
      - "mimeType"
      - "size"
      - "labelInfo"
    include_label_ids: ""
    max_results: 64
    shared_drives: false
processor:
  label: ""
  google_drive_search:
    credentials_json: "" # No default (optional)
    query: "" # No default (required)
    projection:
      - "id"
      - "name"
      - "mimeType"
      - "size"
      - "labelInfo"
    include_label_ids: ""
    max_results: 64
    shared_drives: false

Authentication

By default, this processor uses Google Application Default Credentials (ADC) to authenticate with Google APIs.

To set up local ADC authentication, use the following gcloud commands:

  • Authenticate using Application Default Credentials and grant read-only access to your Google Drive.

    gcloud auth application-default login --scopes='openid,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/drive.readonly'

  • Assign a quota project to the Application Default Credentials when using a user account.

    gcloud auth application-default set-quota-project <project-id>

    Replace the <project-id> placeholder with your Google Cloud project ID

To use a service account instead, create a JSON key for the account and add it to the credentials_json field. To access Google Drive files using a service account, either:

  • Explicitly share files with the service account’s email account

  • Use domain-wide delegation to share all files within a Google Workspace

Fields

credentials_json

The JSON key for your service account (optional). If left empty, Application Default Credentials are used.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

include_label_ids

A comma delimited list of label IDs to include in the Google Drive search result.

This field supports interpolation functions.

Type: string

Default: ""

max_results

The maximum number of search results to return.

Type: int

Default: 64

projection[]

Partial fields to include in the Google Drive search result.

Type: array<string>

Default:

- "id"
- "name"
- "mimeType"
- "size"
- "labelInfo"

query

Specify a search query to locate matching files in Google Drive. This field supports:

Type: string

shared_drives

Whether or not to include shared drives in the result.

Type: bool

Default: false

Examples

Search & download files from Google Drive

This examples downloads all the files from Google Drive that are returned in the query

input:
  stdin: {}
pipeline:
  processors:
    - google_drive_search:
        query: "${!content().string()}"
    - mutation: 'meta path = this.name'
    - google_drive_download:
        file_id: "${!this.id}"
        mime_type: "${!this.mimeType}"
output:
  file:
    path: "${!@path}"
    codec: all-bytes