mongodb
Executes a query and creates a message for each document received.
Introduced in version 3.64.0.
Once the documents from the query are exhausted, this input shuts down, allowing the pipeline to gracefully terminate (or the next input in a sequence to execute).
-
Common
-
Advanced
input:
label: ""
mongodb:
url: "" # No default (required)
database: "" # No default (required)
username: ""
password: ""
collection: "" # No default (required)
query: "" # No default (required)
auto_replay_nacks: true
batch_size: 0 # No default (optional)
sort: {} # No default (optional)
limit: 0 # No default (optional)input:
label: ""
mongodb:
url: "" # No default (required)
database: "" # No default (required)
username: ""
password: ""
app_name: benthos
aws:
enabled: false
region: "" # No default (optional)
session_duration: 1h
id: "" # No default (optional)
secret: "" # No default (optional)
token: "" # No default (optional)
role: "" # No default (optional)
role_external_id: "" # No default (optional)
roles: [] # No default (optional)
collection: "" # No default (required)
operation: find
json_marshal_mode: canonical
query: "" # No default (required)
auto_replay_nacks: true
batch_size: 0 # No default (optional)
sort: {} # No default (optional)
limit: 0 # No default (optional)Fields
app_name
The client application name.
Requires version 4.32.0 or later.
Type: string
Default: benthos
auto_replay_nacks
Whether to automatically replay rejected messages (negative acknowledgements, or nacks) at the output level. If the cause of rejections persists, leaving this option enabled can result in back pressure.
Set auto_replay_nacks to false to delete rejected messages. Disabling auto replays can greatly improve memory efficiency of high throughput streams, as the original shape of the data is discarded immediately upon consumption and mutation.
Requires version 4.27.0 or later.
Type: bool
Default: true
aws
AWS IAM authentication using the MONGODB-AWS mechanism, for example against MongoDB Atlas. When enabled, IAM credentials are used instead of a static username and password. Role-derived session credentials are resolved when the component connects and are re-resolved whenever it reconnects. The mongodb processor and cache establish their client once at creation and cannot refresh expiring session credentials, so role, roles and session tokens are rejected for those components; use the ambient credential chain or long-lived access keys with them. For long-running pipelines, prefer the ambient credential chain (leave keys and roles unset), which the driver refreshes automatically.
Requires version 4.106.0 or later.
Type: object
aws.enabled
Enable AWS IAM authentication using the driver-native MONGODB-AWS mechanism. The MongoDB Atlas database user must be created with the AWS IAM authentication type, and connections require TLS. When no static credentials or roles are configured, the ambient AWS credential chain (environment variables, EC2 instance profile, EKS pod role) is used and expiring credentials are refreshed automatically.
Type: bool
Default: false
aws.id
The AWS access key ID to authenticate with. When empty, the default AWS credential chain is used.
Type: string
aws.region
The AWS region used when assuming roles (for STS calls). Only used when role or roles are configured; the ambient and static-key paths ignore it. If no region is specified then the environment default is used.
Type: string
aws.role
Optional AWS IAM role ARN to assume for authentication. Cannot be combined with roles; use the roles array instead when chaining multiple roles.
Type: string
aws.role_external_id
Optional external ID to use when assuming the role set in role. Each entry in roles sets its own external ID.
Type: string
aws.roles[]
Optional array of AWS IAM roles to assume for authentication. Roles are assumed in sequence, each using the credentials of the previous one, enabling chaining for purposes such as cross-account access. Each role can optionally specify an external ID. Cannot be combined with role.
Type: array<object>
aws.secret
The AWS secret access key that pairs with id.
|
This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Secrets. |
Type: string
aws.session_duration
The duration of the STS session requested when assuming roles. AWS requires at least 15 minutes and caps sessions created through role chaining at one hour. Only used when role or roles are configured. For long-running pipelines, prefer the ambient credential chain over a fixed session duration, since the driver refreshes ambient credentials automatically as they near expiry.
Type: string
Default: 1h
aws.token
The AWS session token to use with id and secret. Required only when using short-term credentials.
Type: string
batch_size
A explicit number of documents to batch up before flushing them for processing. Must be greater than 0. Operations: find, aggregate
Requires version 4.26.0 or later.
Type: int
# Examples:
batch_size: 1000json_marshal_mode
The json_marshal_mode setting is optional and controls the format of the output message.
Requires version 4.7.0 or later.
Type: string
Default: canonical
| Option | Summary |
|---|---|
|
A string format that emphasizes type preservation at the expense of readability and interoperability. That is, conversion from canonical to BSON will generally preserve type information except in certain specific cases. |
|
A string format that emphasizes readability and interoperability at the expense of type preservation.That is, conversion from relaxed format to BSON can lose type information. |
limit
An explicit maximum number of documents to return. Operations: find
Requires version 4.26.0 or later.
Type: int
operation
The mongodb operation to perform.
Requires version 4.2.0 or later.
Type: string
Default: find
Options: find, aggregate
password
The password to use for authentication. Used together with username for basic authentication.
|
This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Secrets. |
Type: string
Default: ""
query
Bloblang expression describing MongoDB query.
Type: string
# Examples:
query: |2
root.from = {"$lte": timestamp_unix()}
root.to = {"$gte": timestamp_unix()}