Cloud
Also in Self-Managed

redpanda

Export distributed tracing data to a Redpanda topic, enabling you to monitor and debug your Redpanda Connect pipelines. Traces are exported in OpenTelemetry format as JSON, allowing integration with observability platforms like Jaeger, Grafana Tempo, or custom trace consumers.

  • Common

  • Advanced

tracer:
  redpanda:
    seed_brokers: [] # No default (required)
    topic: otel-traces
    format: json
    schema_registry:
      url: "" # No default (optional)
    service: redpanda-connect
    sampling:
      enabled: false
      ratio: 0 # No default (optional)
tracer:
  redpanda:
    seed_brokers: [] # No default (required)
    client_id: redpanda-connect
    tls:
      enabled: false
      skip_cert_verify: false
      enable_renegotiation: false
      root_cas: ""
      root_cas_file: ""
      client_certs: []
    sasl: [] # No default (optional)
    metadata_max_age: 1m
    request_timeout_overhead: 10s
    conn_idle_timeout: 20s
    tcp:
      connect_timeout: 0s
      keep_alive:
        idle: 15s
        interval: 15s
        count: 9
      tcp_user_timeout: 0s
    partitioner: "" # No default (optional)
    idempotent_write: true
    acks: all
    compression: "" # No default (optional)
    allow_auto_topic_creation: true
    timeout: 10s
    max_message_bytes: 1MiB
    broker_write_max_bytes: 100MiB
    max_buffered_records: 10000
    max_buffered_bytes: "0"
    max_in_flight_requests: 1
    record_retries: 0
    record_delivery_timeout: 0s
    topic: otel-traces
    format: json
    schema_registry:
      url: "" # No default (optional)
      tls:
        skip_cert_verify: false
        enable_renegotiation: false
        root_cas: ""
        root_cas_file: ""
        client_certs: []
      oauth2:
        enabled: false
        client_key: ""
        client_secret: ""
        token_url: ""
        scopes: []
        endpoint_params: {}
      oauth:
        enabled: false
        consumer_key: ""
        consumer_secret: ""
        access_token: ""
        access_token_secret: ""
      basic_auth:
        enabled: false
        username: ""
        password: ""
      jwt:
        enabled: false
        private_key_file: ""
        signing_method: ""
        claims: {}
        headers: {}
    service: redpanda-connect
    tags: {}
    sampling:
      enabled: false
      ratio: 0 # No default (optional)

This tracer automatically captures trace spans as messages flow through your pipeline, recording timing information, component metadata, and error details. Use this to:

  • Track message flow through complex pipelines with multiple processors.

  • Identify performance bottlenecks by analyzing span durations.

  • Debug failures by examining trace context and error details.

  • Monitor pipeline health across distributed Redpanda Connect instances.

  • Correlate activity across multiple services using trace IDs.

The tracer writes to a dedicated Redpanda topic that can be consumed by trace analysis tools. Configure sampling to control trace volume in high-throughput environments.

Fields

acks

The number of acknowledgements the leader broker must receive from ISR brokers before responding to the produce request. When idempotent_write is enabled this must be set to all.

Type: string

Default: all

Option Summary

all

Wait for all in-sync replicas to acknowledge (acks=-1). Required when idempotent_write is enabled.

leader

Wait for the leader broker to acknowledge (acks=1). Messages are lost if the leader fails before replication.

none

Do not wait for any acknowledgement (acks=0). Highest throughput but messages may be lost.

allow_auto_topic_creation

Enables topics to be auto created if they do not exist when fetching their metadata. If set to false, the topic must already exist.

Type: bool

Default: true

broker_write_max_bytes

The maximum number of bytes this output can write to a broker connection in a single write. This field corresponds to Kafka’s socket.request.max.bytes.

Type: string

Default: 100MiB

# Examples:
broker_write_max_bytes: 128MB

# ---

broker_write_max_bytes: 50mib

client_id

An identifier for the client connection. This identifier appears in broker logs and metrics, which helps you identify the Redpanda Connect instance that is connecting.

Type: string

Default: redpanda-connect

compression

Set an explicit compression type (optional). By default, the client uses snappy when the broker supports it, and falls back to none if not.

Type: string

Options: lz4, snappy, gzip, none, zstd

conn_idle_timeout

The approximate amount of time that connections can remain idle before they are closed. In the worst case, a connection can stay idle for up to twice this value. This field accepts Go duration format strings such as 100ms, 1s, or 5s.

Type: string

Default: 20s

format

The serialization format for individual spans in the topic.

Type: string

Default: json

Option Summary

json

Emit in JSON Format

protobuf

Emit in Protobuf Format

schema-registry-json

Emit in JSON Format with Schema Registry encoding

schema-registry-protobuf

Emit in Protobuf Format with Schema Registry encoding

idempotent_write

Enable the idempotent write producer option. When enabled, the producer initializes a producer ID and uses it to guarantee exactly-once semantics per partition, so retries do not produce duplicates. This option requires the IDEMPOTENT_WRITE permission on the CLUSTER resource. Disable this option if the IDEMPOTENT_WRITE permission is unavailable, for example in some managed Kafka services or Redpanda clusters with strict ACLs. Disabling this option only affects retry behavior: duplicates may occur on producer retries, but the pipeline continues to function normally.

Type: bool

Default: true

max_buffered_bytes

The maximum number of bytes the client will buffer in memory before blocking. When this limit is reached, Produce() calls will block until buffered records are delivered. Set to 0 to disable the byte-level limit (only max_buffered_records applies). This limit is checked after max_buffered_records.

Type: string

Default: 0

# Examples:
max_buffered_bytes: 256MB

# ---

max_buffered_bytes: 50mib

max_buffered_records

The maximum number of records the client will buffer in memory before blocking. When this limit is reached, Produce() calls will block until buffered records are delivered and space frees up. Increase this value for high-throughput pipelines to avoid back-pressure stalls.

Type: int

Default: 10000

max_in_flight_requests

The maximum number of produce requests in flight per broker connection. While idempotent_write is enabled (the default) this must be 1, as the client relies on a single in-flight request per broker to guarantee ordering. To use higher values you must set idempotent_write to false, which allows requests to be pipelined for throughput but may cause duplicate and out-of-order delivery on retries. Note that this is distinct from the output’s max_in_flight field, which counts message batches being written in parallel rather than produce requests on the wire. A value of 1 is not a throughput ceiling: records from concurrent writes are coalesced into fewer, larger produce requests.

Type: int

Default: 1

max_message_bytes

The maximum size of a produced record batch in bytes. A MESSAGE_TOO_LARGE error is returned if a batch exceeds this limit. This field maps to the max.message.bytes Kafka property. Ensure the Redpanda broker’s kafka_batch_max_bytes property is at least as large as this value, see kafka_batch_max_bytes.

Type: string

Default: 1MiB

# Examples:
max_message_bytes: 100MB

# ---

max_message_bytes: 50mib

metadata_max_age

The maximum period of time after which metadata is refreshed. This field accepts Go duration format strings such as 100ms, 1s, or 5s.

Lower values provide more responsive topic and partition discovery but may increase broker load. Higher values reduce broker queries but can delay detection of topology changes.

This interval also controls how frequently regex topic patterns are re-evaluated to discover new matching topics.

Type: string

Default: 1m

partitioner

Override the default murmur2 hashing partitioner.

Type: string

Option Summary

least_backup

Chooses the least backed up partition (the partition with the fewest amount of buffered records). Partitions are selected per batch.

manual

Manually select a partition for each message, requires the field partition to be specified.

murmur2_hash

Kafka’s default hash algorithm that uses a 32-bit murmur2 hash of the key to compute which partition the record will be on.

round_robin

Round-robin’s messages through all available partitions. This algorithm has lower throughput and causes higher CPU load on brokers, but can be useful if you want to ensure an even distribution of records to partitions.

record_delivery_timeout

The maximum time a record can sit in the producer buffer before it is failed, roughly equivalent to Kafka’s delivery.timeout.ms. This is evaluated before writing a request or after a produce response. When a record times out, all records in the same partition are also failed. Set to 0s for no timeout (the default). With idempotent_write enabled, timeouts are only enforced when safe to do so without creating invalid sequence numbers.

Type: string

Default: 0s

record_retries

The maximum number of times a record produce is retried on failure before the record is failed. When a record fails, all records buffered in the same partition are also failed to preserve gapless ordering. Set to 0 for unlimited retries (the default). With idempotent_write enabled, retries are only enforced when safe to do so without creating invalid sequence numbers.

Type: int

Default: 0

request_timeout_overhead

Additional time to apply as overhead when calculating request deadlines. For most requests, the deadline is this overhead alone. For requests that define their own timeout field, the overhead is added on top of that timeout, which helps prevent premature timeouts.

This field is roughly equivalent to Apache Kafka’s request.timeout.ms parameter, but grants extra time to requests that have timeout fields.

Type: string

Default: 10s

sampling

Configure trace sampling to control the volume of trace data. Sampling is recommended for high-volume production workloads to prevent trace data from overwhelming your observability infrastructure.

Type: object

sampling.enabled

Whether to enable trace sampling. When disabled, all traces are exported. When enabled, traces are sampled according to the configured ratio.

Type: bool

Default: false

sampling.ratio

The sampling ratio as a decimal between 0 and 1. For example, 0.1 samples 10% of traces, 0.01 samples 1%. Lower ratios reduce trace volume and overhead. For high-throughput production systems, start with 0.01-0.1 and adjust based on your needs.

Type: float

# Examples:
ratio: 0.05

# ---

ratio: 0.85

# ---

ratio: 0.5

sasl[]

Specify one or more methods or mechanisms of SASL authentication. They are tried in order. If the broker supports the first SASL mechanism, all connections use it. If the first mechanism fails, the client picks the first supported mechanism. If the broker does not support any client mechanisms, all connections fail.

Type: array<object>

# Examples:
sasl:
  - mechanism: SCRAM-SHA-512
    password: bar
    username: foo

sasl[].aws

Contains AWS-specific fields for when sasl.mechanism is set to AWS_MSK_IAM.

Type: object

sasl[].aws.credentials

Manually configure the AWS credentials to use (optional). For more information, see the Amazon Web Services guide.

Type: object

sasl[].aws.credentials.from_ec2_role

Use the credentials of a host EC2 machine configured to assume an IAM role associated with the instance.

Type: bool

sasl[].aws.credentials.id

The ID of the AWS credentials to use.

Type: string

sasl[].aws.credentials.profile

The profile from ~/.aws/credentials to use.

Type: string

sasl[].aws.credentials.role

The ARN of the role to assume.

Type: string

sasl[].aws.credentials.role_external_id

An external ID to use when assuming a role.

Type: string

sasl[].aws.credentials.secret

The secret for the AWS credentials in use.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

sasl[].aws.credentials.token

The token for the AWS credentials in use. Required only when using short-term credentials.

Type: string

sasl[].aws.endpoint

A custom endpoint URL for AWS API requests. Use this to connect to AWS-compatible services or local testing environments instead of the standard AWS endpoints.

Type: string

sasl[].aws.region

The AWS region in which your resources are hosted.

Type: string

sasl[].aws.tcp

Configure TCP socket-level settings to optimize network performance and reliability. These low-level controls are useful for:

  • Unresponsive hosts: Set connect_timeout to limit how long a connection attempt can take (the default 0s sets no limit)

  • Long-lived connections: Configure keep_alive settings to detect and recover from stale connections

  • Unstable networks: Tune keep-alive probes to balance between quick failure detection and avoiding false positives

  • Linux systems with specific requirements: Use tcp_user_timeout (Linux 2.6.37+) to control data acknowledgment timeouts

Most users should keep the default values. Only modify these settings if you’re experiencing connection stability issues or have specific network requirements.

Type: object

sasl[].aws.tcp.connect_timeout

Maximum amount of time a dial will wait for a connect to complete. Zero disables.

Type: string

Default: 0s

sasl[].aws.tcp.keep_alive

TCP keep-alive probe configuration.

Type: object

sasl[].aws.tcp.keep_alive.count

Maximum unanswered keep-alive probes before dropping the connection. Zero defaults to 9.

Type: int

Default: 9

sasl[].aws.tcp.keep_alive.idle

Duration the connection must be idle before sending the first keep-alive probe. Zero defaults to 15s. Negative values disable keep-alive probes.

Type: string

Default: 15s

sasl[].aws.tcp.keep_alive.interval

Duration between keep-alive probes. Zero defaults to 15s.

Type: string

Default: 15s

sasl[].aws.tcp.tcp_user_timeout

Maximum time to wait for acknowledgment of transmitted data before killing the connection. Linux-only (kernel 2.6.37+), ignored on other platforms. When enabled, keep_alive.idle must be greater than this value per RFC 5482. Zero disables.

Type: string

Default: 0s

sasl[].extensions

Key/value pairs to add to OAUTHBEARER authentication requests.

Type: object<string>

sasl[].mechanism

The SASL mechanism to use for authentication.

Type: string

Option Summary

AWS_MSK_IAM

AWS IAM based authentication as specified by the 'aws-msk-iam-auth' java library.

OAUTHBEARER

OAuth Bearer based authentication.

PLAIN

Plain text authentication.

REDPANDA_CLOUD_SERVICE_ACCOUNT

Redpanda Cloud Service Account authentication when running in Redpanda Cloud.

SCRAM-SHA-256

SCRAM based authentication as specified in RFC5802.

SCRAM-SHA-512

SCRAM based authentication as specified in RFC5802.

none

Disable sasl authentication

sasl[].password

The password to use for PLAIN or SCRAM-* authentication.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

sasl[].token

The token to use for a single session’s OAUTHBEARER authentication.

Type: string

Default: ""

sasl[].username

The username to use for PLAIN or SCRAM-* authentication.

Type: string

Default: ""

schema_registry

Schema registry information to publish schemas for tracing data along with the data.

Type: object

schema_registry.basic_auth

Configure basic authentication for requests from this component.

Type: object

schema_registry.basic_auth.enabled

Whether to use basic authentication in requests.

Type: bool

Default: false

schema_registry.basic_auth.password

The password to use for authentication. Used together with username for basic authentication.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

schema_registry.basic_auth.username

The username of the account credentials to authenticate as. Used together with password for basic authentication.

Type: string

Default: ""

schema_registry.jwt

Beta

Configure JSON Web Token (JWT) authentication. This feature is in beta and may change in future releases. JWTs provide secure, stateless authentication between services.

Type: object

schema_registry.jwt.claims

A map of claims to include in the JWT. Claims pass the identity of the authenticated entity to the service provider.

Type: object

Default: {}

schema_registry.jwt.enabled

Whether to use JWT authentication in requests.

Type: bool

Default: false

schema_registry.jwt.headers

Additional key-value pairs to include in the JWT header (optional). These headers provide extra metadata for JWT processing.

Type: object

Default: {}

schema_registry.jwt.private_key_file

Path to a file containing the PEM-encoded private key using PKCS#1 or PKCS#8 format. The private key must be compatible with the algorithm specified in the signing_method field.

Type: string

Default: ""

schema_registry.jwt.signing_method

The cryptographic algorithm used to sign the JWT. Supported algorithms are RS256, RS384, RS512, and EdDSA. This algorithm must be compatible with the private key specified in the private_key_file field.

Type: string

Default: ""

schema_registry.oauth

Configure OAuth version 1.0 authentication for secure API access.

Type: object

schema_registry.oauth.access_token

The value used to gain access to the protected resources on behalf of the user.

Type: string

Default: ""

schema_registry.oauth.access_token_secret

The secret that establishes ownership of the access_token in OAuth 1.0 authentication.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

schema_registry.oauth.consumer_key

The value used to identify this component or client to the service provider.

Type: string

Default: ""

schema_registry.oauth.consumer_secret

The secret that establishes ownership of the consumer key in OAuth 1.0 authentication.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

schema_registry.oauth.enabled

Whether to enable OAuth version 1.0 authentication for requests.

Type: bool

Default: false

schema_registry.oauth2

Allows you to specify open authentication via OAuth version 2 using the client credentials token flow.

Type: object

schema_registry.oauth2.client_key

A value used to identify the client to the token provider.

Type: string

Default: ""

schema_registry.oauth2.client_secret

A secret used to establish ownership of the client key.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

schema_registry.oauth2.enabled

Whether to use OAuth version 2 in requests.

Type: bool

Default: false

schema_registry.oauth2.endpoint_params

A list of optional endpoint parameters, values should be arrays of strings.

Type: object

Default: {}

# Examples:
endpoint_params:
  audience:
    - https://example.com
  resource:
    - https://api.example.com

schema_registry.oauth2.scopes[]

A list of optional requested permissions.

Type: array<string>

Default: []

schema_registry.oauth2.token_url

The URL of the token provider.

Type: string

Default: ""

schema_registry.tls

Configure Transport Layer Security (TLS) settings to secure network connections. This includes options for standard TLS as well as mutual TLS (mTLS) authentication where both client and server authenticate each other using certificates. Key configuration options include client_certs for mTLS authentication, root_cas/root_cas_file for custom certificate authorities, and skip_cert_verify for development environments.

Type: object

schema_registry.tls.client_certs[]

A list of client certificates for mutual TLS (mTLS) authentication. Configure this field to enable mTLS, authenticating the client to the server with these certificates.

Certificate pairing rules: For each certificate item, provide either:

  • Inline PEM data using both cert and key or

  • File paths using both cert_file and key_file.

Mixing inline and file-based values within the same item is not supported.

Type: array<object>

Default: []

# Examples:
client_certs:
  - cert: foo
    key: bar


# ---

client_certs:
  - cert_file: ./example.pem
    key_file: ./example.key

schema_registry.tls.client_certs[].cert

The plaintext certificate to use for TLS authentication. Must be paired with the corresponding private key in the key field when using inline PEM data for mTLS client certificates.

Type: string

Default: ""

schema_registry.tls.client_certs[].cert_file

The path to a file containing the certificate to use for TLS authentication. Must be paired with the corresponding private key file in the key_file field when using file-based configuration for mTLS client certificates.

Type: string

Default: ""

schema_registry.tls.client_certs[].key

Private key for mTLS client certificate as inline PEM data. Must correspond to the client certificate specified in the cert field. Use this field together with cert when providing certificate data inline rather than through files.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

schema_registry.tls.client_certs[].key_file

Path to private key file for mTLS client certificate in PEM format. Must correspond to the client certificate specified in the cert_file field. Use this field together with cert_file when loading certificate data from files.

Type: string

Default: ""

schema_registry.tls.client_certs[].password

The password to use for the private key (specified in the key or key_file fields), if it is password-protected. The PKCS#1 and PKCS#8 formats are supported. Supports environment variable interpolation for secure password management.

The pbeWithMD5AndDES-CBC algorithm is obsolete and not supported for the PKCS#8 format. This algorithm does not authenticate the ciphertext, making it vulnerable to padding oracle attacks that can let an attacker recover the plaintext.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

# Examples:
password: foo

# ---

password: ${KEY_PASSWORD}

schema_registry.tls.enable_renegotiation

Whether to allow the remote server to repeatedly request renegotiation. Enable this option if you’re seeing the error message local error: tls: no renegotiation.

Type: bool

Default: false

schema_registry.tls.root_cas

Specify a root certificate authority to use (optional). This is a string that represents a certificate chain from the parent-trusted root certificate, through possible intermediate signing certificates, to the host certificate. Use either this field for inline certificate data or root_cas_file for file-based certificate loading.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

# Examples:
root_cas: |-
  -----BEGIN CERTIFICATE-----
  ...
  -----END CERTIFICATE-----

schema_registry.tls.root_cas_file

Specify the path to a root certificate authority file (optional). This is a file, often with a .pem extension, which contains a certificate chain from the parent-trusted root certificate, through possible intermediate signing certificates, to the host certificate. Use either this field for file-based certificate loading or root_cas for inline certificate data.

Type: string

Default: ""

# Examples:
root_cas_file: ./root_cas.pem

schema_registry.tls.skip_cert_verify

Whether to skip server-side certificate verification. Set to true only for testing environments as this reduces security by disabling certificate validation. When using self-signed certificates or in development, this may be necessary, but should never be used in production. Consider using root_cas or root_cas_file to specify trusted certificates instead of disabling verification entirely.

Type: bool

Default: false

schema_registry.url

The base URL of the schema registry service.

Type: string

seed_brokers[]

A list of broker addresses used to establish connections. If an item of the list contains commas, it is expanded into multiple addresses.

Type: array<string>

# Examples:
seed_brokers:
  - "localhost:9092"

# ---

seed_brokers:
  - "foo:9092"
  - "bar:9092"

# ---

seed_brokers:
  - "foo:9092,bar:9092"

service

The service name to identify this Redpanda Connect instance in traces. This appears in trace visualizations and helps correlate traces across distributed systems. Use descriptive names like order-processor or analytics-pipeline.

Type: string

Default: redpanda-connect

tags

Custom key-value tags to attach to all traces from this instance. Use tags to add metadata like environment (production, staging), region, version, or instance identifiers. Tags appear as resource attributes in OpenTelemetry traces.

Type: object<string>

Default: {}

tcp

Configure TCP socket-level settings to optimize network performance and reliability. These low-level controls are useful for:

  • Unresponsive hosts: Set connect_timeout to limit how long a connection attempt can take (the default 0s sets no limit)

  • Long-lived connections: Configure keep_alive settings to detect and recover from stale connections

  • Unstable networks: Tune keep-alive probes to balance between quick failure detection and avoiding false positives

  • Linux systems with specific requirements: Use tcp_user_timeout (Linux 2.6.37+) to control data acknowledgment timeouts

Most users should keep the default values. Only modify these settings if you’re experiencing connection stability issues or have specific network requirements.

Type: object

tcp.connect_timeout

Maximum amount of time a dial will wait for a connect to complete. Zero disables.

Type: string

Default: 0s

tcp.keep_alive

TCP keep-alive probe configuration.

Type: object

tcp.keep_alive.count

Maximum unanswered keep-alive probes before dropping the connection. Zero defaults to 9.

Type: int

Default: 9

tcp.keep_alive.idle

Duration the connection must be idle before sending the first keep-alive probe. Zero defaults to 15s. Negative values disable keep-alive probes.

Type: string

Default: 15s

tcp.keep_alive.interval

Duration between keep-alive probes. Zero defaults to 15s.

Type: string

Default: 15s

tcp.tcp_user_timeout

Maximum time to wait for acknowledgment of transmitted data before killing the connection. Linux-only (kernel 2.6.37+), ignored on other platforms. When enabled, keep_alive.idle must be greater than this value per RFC 5482. Zero disables.

Type: string

Default: 0s

timeout

The maximum period of time to wait for message sends before abandoning the request and retrying.

Type: string

Default: 10s

tls

Configure Transport Layer Security (TLS) settings to secure network connections. This includes options for standard TLS as well as mutual TLS (mTLS) authentication where both client and server authenticate each other using certificates. Key configuration options include enabled to enable TLS, client_certs for mTLS authentication, root_cas/root_cas_file for custom certificate authorities, and skip_cert_verify for development environments.

Type: object

tls.client_certs[]

A list of client certificates for mutual TLS (mTLS) authentication. Configure this field to enable mTLS, authenticating the client to the server with these certificates.

You must set tls.enabled: true for the client certificates to take effect.

Certificate pairing rules: For each certificate item, provide either:

  • Inline PEM data using both cert and key or

  • File paths using both cert_file and key_file.

Mixing inline and file-based values within the same item is not supported.

Type: array<object>

Default: []

# Examples:
client_certs:
  - cert: foo
    key: bar


# ---

client_certs:
  - cert_file: ./example.pem
    key_file: ./example.key

tls.client_certs[].cert

The plaintext certificate to use for TLS authentication. Must be paired with the corresponding private key in the key field when using inline PEM data for mTLS client certificates.

Type: string

Default: ""

tls.client_certs[].cert_file

The path to a file containing the certificate to use for TLS authentication. Must be paired with the corresponding private key file in the key_file field when using file-based configuration for mTLS client certificates.

Type: string

Default: ""

tls.client_certs[].key

Private key for mTLS client certificate as inline PEM data. Must correspond to the client certificate specified in the cert field. Use this field together with cert when providing certificate data inline rather than through files.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

tls.client_certs[].key_file

Path to private key file for mTLS client certificate in PEM format. Must correspond to the client certificate specified in the cert_file field. Use this field together with cert_file when loading certificate data from files.

Type: string

Default: ""

tls.client_certs[].password

The password to use for the private key (specified in the key or key_file fields), if it is password-protected. The PKCS#1 and PKCS#8 formats are supported. Supports environment variable interpolation for secure password management.

The pbeWithMD5AndDES-CBC algorithm is obsolete and not supported for the PKCS#8 format. This algorithm does not authenticate the ciphertext, making it vulnerable to padding oracle attacks that can let an attacker recover the plaintext.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

# Examples:
password: foo

# ---

password: ${KEY_PASSWORD}

tls.enable_renegotiation

Whether to allow the remote server to repeatedly request renegotiation. Enable this option if you’re seeing the error message local error: tls: no renegotiation.

Type: bool

Default: false

tls.enabled

Whether to enable TLS for secure connections. Set to true to enable TLS encryption. Required to be true for other TLS options (like client_certs, root_cas, etc.) to take effect.

Type: bool

Default: false

tls.root_cas

Specify a root certificate authority to use (optional). This is a string that represents a certificate chain from the parent-trusted root certificate, through possible intermediate signing certificates, to the host certificate. Use either this field for inline certificate data or root_cas_file for file-based certificate loading.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

# Examples:
root_cas: |-
  -----BEGIN CERTIFICATE-----
  ...
  -----END CERTIFICATE-----

tls.root_cas_file

Specify the path to a root certificate authority file (optional). This is a file, often with a .pem extension, which contains a certificate chain from the parent-trusted root certificate, through possible intermediate signing certificates, to the host certificate. Use either this field for file-based certificate loading or root_cas for inline certificate data.

Type: string

Default: ""

# Examples:
root_cas_file: ./root_cas.pem

tls.skip_cert_verify

Whether to skip server-side certificate verification. Set to true only for testing environments as this reduces security by disabling certificate validation. When using self-signed certificates or in development, this may be necessary, but should never be used in production. Consider using root_cas or root_cas_file to specify trusted certificates instead of disabling verification entirely.

Type: bool

Default: false

topic

The Redpanda topic where trace data is written. This topic should be dedicated to traces and configured with appropriate retention policies. Default: otel-traces

Type: string

Default: otel-traces

Examples

Basic tracing setup

Enable tracing for a pipeline, sending all trace data to a local Redpanda cluster.

input:
  redpanda:
    seed_brokers: [ "localhost:9092" ]
    topics: [ "orders" ]
    consumer_group: order-processor

pipeline:
  processors:
    - mapping: |
        root = this
        root.processed_at = now()

output:
  redpanda:
    seed_brokers: [ "localhost:9092" ]
    topic: processed_orders

tracer:
  redpanda:
    seed_brokers: [ "localhost:9092" ]
    topic: otel-traces
    service: order-processor

Production tracing with sampling

Configure tracing for a production environment with sampling to reduce trace volume, and with TLS and SASL authentication.

input:
  redpanda:
    seed_brokers: [ "redpanda-prod:9092" ]
    topics: [ "events" ]
    consumer_group: event-processor

pipeline:
  processors:
    - branch:
        request_map: 'root = this'
        processors:
          - mapping: 'root.enriched = true'

output:
  redpanda:
    seed_brokers: [ "redpanda-prod:9092" ]
    topic: processed_events

tracer:
  redpanda:
    seed_brokers: [ "redpanda-prod:9092" ]
    topic: otel-traces
    service: event-processor-prod
    sampling:
      enabled: true
      ratio: 0.1 # Sample 10% of traces
    tls:
      enabled: true
    sasl:
      - mechanism: SCRAM-SHA-512
        username: tracer-user
        password: ${TRACER_PASSWORD}

Multi-instance tracing

Configure tracing for multiple Redpanda Connect instances with unique service names for correlation. Each instance writes to the same topic and sets its own service name and instance tag. This configuration is for an ingestion service. A processing service uses the same configuration with service: processing-service and instance: processing-01.

tracer:
  redpanda:
    seed_brokers: [ "redpanda:9092" ]
    topic: otel-traces
    service: ingestion-service
    tags:
      environment: production
      region: us-west-2
      instance: ingestion-01