Cloud
beta
Also in Self-Managed

timeplus

Sends message to a Timeplus Enterprise stream via ingest endpoint

This output can send message to Timeplus Enterprise Cloud, Timeplus Enterprise (self-hosted) or directly to timeplusd.

This output accepts structured message only. It also expects all message contains the same keys and matches the schema of the destination stream. If the upstream source or pipeline returns unstructured message such as string, please refer to the "Unstructured message" example.

  • Common

  • Advanced

output:
  label: ""
  timeplus:
    target: timeplus
    url: https://us-west-2.timeplus.cloud
    workspace: "" # No default (optional)
    stream: "" # No default (required)
    apikey: "" # No default (optional)
    username: "" # No default (optional)
    password: "" # No default (optional)
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
output:
  label: ""
  timeplus:
    target: timeplus
    url: https://us-west-2.timeplus.cloud
    workspace: "" # No default (optional)
    stream: "" # No default (required)
    apikey: "" # No default (optional)
    username: "" # No default (optional)
    password: "" # No default (optional)
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
      processors: [] # No default (optional)

Examples

To Timeplus Enterprise Cloud

You will need to create API Key on Timeplus Enterprise Cloud Web console first and then set the apikey field.

output:
  timeplus:
    workspace: my_workspace_id
    stream: mystream
    apikey: <Your API Key>

To Timeplus Enterprise (self-hosted)

For self-hosted Timeplus Enterprise, you will need to specify the username and password as well as the URL of the App server

output:
  timeplus:
    url: http://localhost:8000
    workspace: my_workspace_id
    stream: mystream
    username: username
    password: pw

To Timeplusd

This output writes to Timeplusd via HTTP so make sure you specify the HTTP port of the Timeplusd.

output:
  timeplus:
    url: http://localhost:3218
    stream: mystream
    username: username
    password: pw

Unstructured message

If the upstream source or pipeline returns unstructured message such as string, you can leverage the output processors to wrap it into a structured message and then pass it to the output. This example create a structured message with raw field and store the original string content into this field. You can modify the name of this raw field to whatever you want. Please make sure the destination stream contains such field

output:
  timeplus:
    workspace: my_workspace_id
    stream: mystream
    apikey: <Api key generated on web console>

  processors:
    - mapping: |
        root = {}
        root.raw = content().string()

Fields

apikey

The API key. Required if you are sending message to Timeplus Enterprise Cloud

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

batching

Configure a batching policy.

Type: object

# Examples:
batching:
  byte_size: 5000
  count: 0
  period: 1s

# ---

batching:
  count: 10
  period: 1s

# ---

batching:
  check: this.contains("END BATCH")
  count: 0
  period: 1m

batching.byte_size

The maximum total size (in bytes) that a batch can reach before it is flushed. When the combined size of all messages in the batch reaches or exceeds this limit, the batch is immediately sent to the next stage (such as a processor or output).

Set to 0 to disable size-based batching. When disabled, messages are flushed based on other conditions (such as count or period).

Type: int

Default: 0

batching.check

A Bloblang query that returns a boolean value indicating whether a message should end a batch.

Type: string

Default: ""

# Examples:
check: this.type == "end_of_transaction"

batching.count

The number of messages at which the batch is flushed. Set to 0 to disable count-based batching.

Type: int

Default: 0

batching.period

The length of time after which an incomplete batch is flushed regardless of its size. This field accepts Go duration format strings such as 100ms, 1s, or 5s. Supported time units are ns, us, ms, s, m, and h.

Type: string

Default: ""

# Examples:
period: 1s

# ---

period: 1m

# ---

period: 500ms

batching.processors[]

A list of processors to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. All resulting messages are flushed as a single batch, so splitting the batch into smaller batches with these processors has no effect.

Type: array<processor>

# Examples:
processors:
  - archive:
      format: concatenate


# ---

processors:
  - archive:
      format: lines


# ---

processors:
  - archive:
      format: json_array

max_in_flight

The maximum number of messages to have in flight at a given time. For outputs that send messages in batches, this limit applies to message batches. Increase this value to improve throughput.

Type: int

Default: 64

password

The password. Required if you are sending message to Timeplus Enterprise (self-hosted) or timeplusd

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

stream

The name of the stream. Make sure the schema of the stream matches the input

Type: string

target

The destination type, either Timeplus Enterprise or timeplusd

Type: string

Default: timeplus

Options: timeplus, timeplusd

url

The url should always include schema and host.

Type: string

Default: https://us-west-2.timeplus.cloud

# Examples:
url: http://localhost:8000

# ---

url: http://127.0.0.1:3218

username

The username. Required if you are sending message to Timeplus Enterprise (self-hosted) or timeplusd

Type: string

workspace

ID of the workspace. Required if target is timeplus.

Type: string