Connect
Self-Managed Only

subprocess

This component is in beta. Its behavior might change in a future release.

Executes a command, runs it as a subprocess, and consumes messages from it over stdout.

Messages are consumed according to a specified codec. The command is executed once and if it terminates the input also closes down gracefully. Alternatively, the field restart_on_close can be set to true in order to have Redpanda Connect re-execute the command each time it stops.

The field max_buffer defines the maximum message size able to be read from the subprocess. This value should be set significantly above the real expected maximum message size.

The execution environment of the subprocess is the same as the Redpanda Connect instance, including environment variables and the current working directory.

  • Common

  • Advanced

input:
  label: ""
  subprocess:
    name: "" # No default (required)
    args: []
    codec: lines
    restart_on_exit: false
input:
  label: ""
  subprocess:
    name: "" # No default (required)
    args: []
    codec: lines
    restart_on_exit: false
    max_buffer: 65536

Fields

args[]

A list of arguments to provide the command.

Type: array<string>

Default: []

codec

The way in which messages should be consumed from the subprocess.

Type: string

Default: lines

Options: lines

max_buffer

The maximum expected size of an individual message.

Type: int

Default: 65536

name

The command to execute as a subprocess.

Type: string

# Examples:
name: cat

# ---

name: sed

# ---

name: awk

restart_on_exit

Whether the command should be re-executed each time the subprocess ends.

Type: bool

Default: false