Cloud
Also in Self-Managed

gcp_pubsub

Consumes messages from a GCP Cloud Pub/Sub subscription.

For information on how to set up credentials see this guide.

Metadata

This input adds the following metadata fields to each message:

  • gcp_pubsub_publish_time_unix - The time at which the message was published to the topic.

  • gcp_pubsub_delivery_attempt - When dead lettering is enabled, this is set to the number of times PubSub has attempted to deliver a message.

  • gcp_pubsub_message_id - The unique identifier of the message.

  • gcp_pubsub_ordering_key - The ordering key of the message.

  • All message attributes

You can access these metadata fields using function interpolation.

  • Common

  • Advanced

input:
  label: ""
  gcp_pubsub:
    project: "" # No default (required)
    credentials_json: ""
    subscription: "" # No default (required)
    endpoint: ""
    sync: false
    max_outstanding_messages: 1000
    max_outstanding_bytes: 1000000000
input:
  label: ""
  gcp_pubsub:
    project: "" # No default (required)
    credentials_json: ""
    subscription: "" # No default (required)
    endpoint: ""
    sync: false
    max_outstanding_messages: 1000
    max_outstanding_bytes: 1000000000
    create_subscription:
      enabled: false
      topic: ""

Fields

create_subscription

Allows you to configure the input subscription and creates if it doesn’t exist.

Type: object

create_subscription.enabled

Whether to configure subscription or not.

Type: bool

Default: false

create_subscription.topic

Defines the topic that the subscription should be vinculated to.

Type: string

Default: ""

credentials_json

The Google Service Account credentials in JSON format (optional). Provide the contents of the credentials file as plain JSON, not Base64-encoded. Use this field to authenticate with Google Cloud services. If this field is empty, the component uses Application Default Credentials. For more information about creating service account credentials, see Google’s service account documentation.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

endpoint

An optional endpoint that overrides the default of pubsub.googleapis.com:443. Use this field to connect to a region-specific Pub/Sub endpoint. For a list of valid values, see Pub/Sub regional endpoints.

Type: string

Default: ""

# Examples:
endpoint: us-central1-pubsub.googleapis.com:443

# ---

endpoint: us-west3-pubsub.googleapis.com:443

max_outstanding_bytes

The maximum number of outstanding pending messages to be consumed measured in bytes.

Type: int

Default: 1000000000

max_outstanding_messages

The maximum number of outstanding pending messages to be consumed at a given time.

Type: int

Default: 1000

project

The project ID of the target subscription.

Type: string

subscription

The target subscription ID.

Type: string

sync

Enable synchronous pull mode.

Type: bool

Default: false