Cloud

Bloblang Methods

Methods provide most of the power in Bloblang as they allow you to augment values and can be added to any expression (including other methods):

root.doc.id = this.thing.id.string().catch(uuid_v4())
root.doc.reduced_nums = this.thing.nums.map_each(num -> if num < 10 {
  deleted()
} else {
  num - 10
})
root.has_good_taste = ["pikachu","mewtwo","magmar"].contains(this.user.fav_pokemon)

# In:  {"thing":{"id":123,"nums":[5,12,18,7,25]},"user":{"fav_pokemon":"pikachu"}}

Methods support both named and nameless style arguments:

root.foo_one = this.(bar | baz).trim().replace_all(old: "dog", new: "cat")
root.foo_two = this.(bar | baz).trim().replace_all("dog", "cat")

# In:  {"bar":"  I love my dog  "}

General

apply

Apply a declared mapping to a target value.

Parameters

Name Type Description

mapping

string

The mapping to apply.

Examples

map thing {
  root.inner = this.first
}

root.foo = this.doc.apply("thing")

# In:  {"doc":{"first":"hello world"}}
# Out: {"foo":{"inner":"hello world"}}
map create_foo {
  root.name = "a foo"
  root.purpose = "to be a foo"
}

root = this
root.foo = null.apply("create_foo")

# In:  {"id":"1234"}
# Out: {"foo":{"name":"a foo","purpose":"to be a foo"},"id":"1234"}

catch

If the result of a target query fails (due to incorrect types, failed parsing, etc) the argument is returned instead.

Parameters

Name Type Description

fallback

query expression

A value to yield, or query to execute, if the target query fails.

Examples

root.doc.id = this.thing.id.string().catch(uuid_v4())

The fallback argument can be a mapping, allowing you to capture the error string and yield structured data back:

root.url = this.url.parse_url().catch(err -> {"error":err,"input":this.url})

# In:  {"url":"invalid %&# url"}
# Out: {"url":{"error":"field `this.url`: parse \"invalid %&\": invalid URL escape \"%&\"","input":"invalid %&# url"}}

When the input document is not structured attempting to reference structured fields with this will result in an error. Therefore, a convenient way to delete non-structured data is with a catch:

root = this.catch(deleted())

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":{"foo":"bar"}}

# In:  not structured data
# Out: <Message deleted>

from

Modifies a target query such that certain functions are executed from the perspective of another message in the batch. This allows you to mutate events based on the contents of other messages. Functions that support this behavior are content, json and meta.

Parameters

Name Type Description

index

integer

The message index to use as a perspective.

Examples

For example, the following map extracts the contents of the JSON field foo specifically from message index 1 of a batch, effectively overriding the field foo for all messages of a batch to that of message 1:

root = this
root.foo = json("foo").from(1)

from_all

Modifies a target query such that certain functions are executed from the perspective of each message in the batch, and returns the set of results as an array. Functions that support this behavior are content, json and meta.

Examples

root = this
root.foo_summed = json("foo").from_all().sum()

map

Executes a query on the target value, allowing you to transform or extract data from the current context.

Parameters

Name Type Description

query

query expression

A query to execute on the target.

not

Returns the logical NOT (negation) of a boolean value. Converts true to false and false to true.

or

If the result of the target query fails or resolves to null, returns the argument instead. This is an explicit method alternative to the coalesce pipe operator |.

Parameters

Name Type Description

fallback

query expression

A value to yield, or query to execute, if the target query fails or resolves to null.

Examples

root.doc.id = this.thing.id.or(uuid_v4())

Encoding and encryption

compress

Compresses a string or byte array using the specified compression algorithm. Returns compressed data as bytes. Useful for reducing payload size before transmission or storage.

Parameters

Name Type Description

algorithm

string

The compression algorithm: flate, gzip, pgzip (parallel gzip), lz4, snappy, zlib, or zstd.

level (optional)

integer

Compression level (default: -1 for default compression). Higher values increase compression ratio but use more CPU. Range and effect varies by algorithm. Default: -1.

Examples

Compress and encode for safe transmission:

root.compressed = content().bytes().compress("gzip").encode("base64")

# In:  {"message":"hello world I love space"}
# Out: {"compressed":"H4sIAAAJbogA/wAmANn/eyJtZXNzYWdlIjoiaGVsbG8gd29ybGQgSSBsb3ZlIHNwYWNlIn0DAHEvdwomAAAA"}

Compare compression ratios across algorithms:

root.original_size = content().length()
root.gzip_size = content().compress("gzip").length()
root.lz4_size = content().compress("lz4").length()

# In:  The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.
# Out: {"gzip_size":114,"lz4_size":85,"original_size":89}

decode

Decodes an encoded string target according to a chosen scheme and returns the result as a byte array. When mapping the result to a JSON field the value should be cast to a string using the method string, or encoded using the method encode, otherwise it will be base64 encoded by default.

Available schemes are: base64, base64url (RFC 4648 with padding characters), base64rawurl (RFC 4648 without padding characters), hex, ascii85.

Parameters

Name Type Description

scheme

string

The decoding scheme to use.

Examples

root.decoded = this.value.decode("hex").string()

# In:  {"value":"68656c6c6f20776f726c64"}
# Out: {"decoded":"hello world"}
root = this.encoded.decode("ascii85")

# In:  {"encoded":"FD,B0+DGm>FDl80Ci\"A>F`)8BEckl6F`M&(+Cno&@/"}
# Out: this is totally unstructured data

decompress

Decompresses a byte array using the specified decompression algorithm. Returns decompressed data as bytes. Use with data that was previously compressed using the corresponding algorithm.

Parameters

Name Type Description

algorithm

string

The decompression algorithm: gzip, pgzip (parallel gzip), zlib, bzip2, flate, snappy, lz4, or zstd.

Examples

Decompress base64-encoded compressed data:

root = this.compressed.decode("base64").decompress("gzip")

# In:  {"compressed":"H4sIAN12MWkAA8tIzcnJVyjPL8pJUfBUyMkvS1UoLkhMTgUAQpDxbxgAAAA="}
# Out: hello world I love space

Convert decompressed bytes to string for JSON output:

root.message = this.compressed.decode("base64").decompress("gzip").string()

# In:  {"compressed":"H4sIAN12MWkAA8tIzcnJVyjPL8pJUfBUyMkvS1UoLkhMTgUAQpDxbxgAAAA="}
# Out: {"message":"hello world I love space"}

decrypt_aes

Decrypts an encrypted string or byte array target according to a chosen AES encryption method and returns the result as a byte array. The algorithms require a key and an initialization vector / nonce. Available schemes are: ctr, gcm, ofb, cbc.

Parameters

Name Type Description

scheme

string

The scheme to use for decryption, one of ctr, gcm, ofb, cbc.

key

string

A key to decrypt with.

iv

string

An initialization vector / nonce.

Examples

let key = "2b7e151628aed2a6abf7158809cf4f3c".decode("hex")
let vector = "f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff".decode("hex")
root.decrypted = this.value.decode("hex").decrypt_aes("ctr", $key, $vector).string()

# In:  {"value":"84e9b31ff7400bdf80be7254"}
# Out: {"decrypted":"hello world!"}

encode

Encodes a string or byte array target according to a chosen scheme and returns a string result. Available schemes are: base64, base64url (RFC 4648 with padding characters), base64rawurl (RFC 4648 without padding characters), hex, ascii85.

Parameters

Name Type Description

scheme

string

The encoding scheme to use.

Examples

root.encoded = this.value.encode("hex")

# In:  {"value":"hello world"}
# Out: {"encoded":"68656c6c6f20776f726c64"}
root.encoded = content().encode("ascii85")

# In:  this is totally unstructured data
# Out: {"encoded":"FD,B0+DGm>FDl80Ci\"A>F`)8BEckl6F`M&(+Cno&@/"}

encrypt_aes

Encrypts a string or byte array target according to a chosen AES encryption method and returns a string result. The algorithms require a key and an initialization vector / nonce. Available schemes are: ctr, gcm, ofb, cbc.

Parameters

Name Type Description

scheme

string

The scheme to use for encryption, one of ctr, gcm, ofb, cbc.

key

string

A key to encrypt with.

iv

string

An initialization vector / nonce.

Examples

let key = "2b7e151628aed2a6abf7158809cf4f3c".decode("hex")
let vector = "f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff".decode("hex")
root.encrypted = this.value.encrypt_aes("ctr", $key, $vector).encode("hex")

# In:  {"value":"hello world!"}
# Out: {"encrypted":"84e9b31ff7400bdf80be7254"}

hash

Hashes a string or byte array according to a chosen algorithm and returns the result as a byte array. When mapping the result to a JSON field the value should be cast to a string using the method string, or encoded using the method encode, otherwise it will be base64 encoded by default.

Available algorithms are: hmac_sha1, hmac_sha256, hmac_sha512, md5, sha1, sha256, sha512, sha3_256, sha3_512, xxhash64, crc32, fnv32.

The following algorithms require a key, which is specified as a second argument: hmac_sha1, hmac_sha256, hmac_sha512.

Parameters

Name Type Description

algorithm

string

The hashing algorithm to use.

key (optional)

string

An optional key to use.

polynomial (optional)

string

An optional polynomial key to use when selecting the crc32 algorithm, otherwise ignored. Options are IEEE (default), Castagnoli and Koopman. Default: "IEEE".

Examples

root.h1 = this.value.hash("sha1").encode("hex")
root.h2 = this.value.hash("hmac_sha1","static-key").encode("hex")

# In:  {"value":"hello world"}
# Out: {"h1":"2aae6c35c94fcfb415dbe95f408b9ce91ee846ed","h2":"d87e5f068fa08fe90bb95bc7c8344cb809179d76"}

The crc32 algorithm supports options for the polynomial:

root.h1 = this.value.hash(algorithm: "crc32", polynomial: "Castagnoli").encode("hex")
root.h2 = this.value.hash(algorithm: "crc32", polynomial: "Koopman").encode("hex")

# In:  {"value":"hello world"}
# Out: {"h1":"c99465aa","h2":"df373d3c"}

uuid_v5

Returns UUID version 5 for the given string.

Parameters

Name Type Description

ns (optional)

string

An optional namespace name or UUID. It supports the dns, url, oid and x500 predefined namespaces and any valid RFC-9562 UUID. If empty, the nil UUID will be used.

Examples

root.id = "example".uuid_v5()
root.id = "example".uuid_v5("x500")
root.id = "example".uuid_v5("77f836b7-9f61-46c0-851e-9b6ca3535e69")

GeoIP

geoip_anonymous_ip

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the anonymous IP associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_asn

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the ASN associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_city

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the city associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_connection_type

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the connection type associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_country

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the country associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_domain

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the domain associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_enterprise

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the enterprise associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_isp

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the ISP associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

JSON web tokens

parse_jwt_es256

Parses a claims object from a JWT string encoded with ES256. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The ES256 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_es256("""-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEGtLqIBePHmIhQcf0JLgc+F/4W/oI
dp0Gta53G35VerNDgUUXmp78J2kfh4qLdh0XtmOMI587tCaqjvDAXfs//w==
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.GIRajP9JJbpTlqSCdNEz4qpQkRvzX4Q51YnTwVyxLDM9tKjR_a8ggHWn9CWj7KG0x8J56OWtmUxn112SRTZVhQ"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_es384

Parses a claims object from a JWT string encoded with ES384. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The ES384 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_es384("""-----BEGIN PUBLIC KEY-----
MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAERoz74/B6SwmLhs8X7CWhnrWyRrB13AuU
8OYeqy0qHRu9JWNw8NIavqpTmu6XPT4xcFanYjq8FbeuM11eq06C52mNmS4LLwzA
2imlFEgn85bvJoC3bnkuq4mQjwt9VxdH
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.H2HBSlrvQBaov2tdreGonbBexxtQB-xzaPL4-tNQZ6TVh7VH8VBcSwcWHYa1lBAHmdsKOFcB2Wk0SB7QWeGT3ptSgr-_EhDMaZ8bA5spgdpq5DsKfaKHrd7DbbQlmxNq"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_es512

Parses a claims object from a JWT string encoded with ES512. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The ES512 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_es512("""-----BEGIN PUBLIC KEY-----
MIGbMBAGByqGSM49AgEGBSuBBAAjA4GGAAQAkHLdts9P56fFkyhpYQ31M/Stwt3w
vpaxhlfudxnXgTO1IP4RQRgryRxZ19EUzhvWDcG3GQIckoNMY5PelsnCGnIBT2Xh
9NQkjWF5K6xS4upFsbGSAwQ+GIyyk5IPJ2LHgOyMSCVh5gRZXV3CZLzXujx/umC9
UeYyTt05zRRWuD+p5bY=
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.ACrpLuU7TKpAnncDCpN9m85nkL55MJ45NFOBl6-nEXmNT1eIxWjiP4pwWVbFH9et_BgN14119jbL_KqEJInPYc9nAXC6dDLq0aBU-dalvNl4-O5YWpP43-Y-TBGAsWnbMTrchILJ4-AEiICe73Ck5yWPleKg9c3LtkEFWfGs7BoPRguZ"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_hs256

Parses a claims object from a JWT string encoded with HS256. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The HS256 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_hs256("""dont-tell-anyone""")

# In:  {"signed":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.YwXOM8v3gHVWcQRRRQc_zDlhmLnM62fwhFYGpiA0J1A"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_hs384

Parses a claims object from a JWT string encoded with HS384. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The HS384 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_hs384("""dont-tell-anyone""")

# In:  {"signed":"eyJhbGciOiJIUzM4NCIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.2Y8rf_ijwN4t8hOGGViON_GrirLkCQVbCOuax6EoZ3nluX0tCGezcJxbctlIfsQ2"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_hs512

Parses a claims object from a JWT string encoded with HS512. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The HS512 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_hs512("""dont-tell-anyone""")

# In:  {"signed":"eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.utRb0urG6LGGyranZJVo5Dk0Fns1QNcSUYPN0TObQ-YzsGGB8jrxHwM5NAJccjJZzKectEUqmmKCaETZvuX4Fg"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_rs256

Parses a claims object from a JWT string encoded with RS256. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The RS256 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_rs256("""-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs/ibN8r68pLMR6gRzg4S
8v8l6Q7yi8qURjkEbcNeM1rkokC7xh0I4JVTwxYSVv/JIW8qJdyspl5NIfuAVi32
WfKvSAs+NIs+DMsNPYw3yuQals4AX8hith1YDvYpr8SD44jxhz/DR9lYKZFGhXGB
+7NqQ7vpTWp3BceLYocazWJgusZt7CgecIq57ycM5hjM93BvlrUJ8nQ1a46wfL/8
Cy4P0et70hzZrsjjN41KFhKY0iUwlyU41yEiDHvHDDsTMBxAZosWjSREGfJL6Mfp
XOInTHs/Gg6DZMkbxjQu6L06EdJ+Q/NwglJdAXM7Zo9rNELqRig6DdvG5JesdMsO
+QIDAQAB
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.b0lH3jEupZZ4zoaly4Y_GCvu94HH6UKdKY96zfGNsIkPZpQLHIkZ7jMWlLlNOAd8qXlsBGP_i8H2qCKI4zlWJBGyPZgxXDzNRPVrTDfFpn4t4nBcA1WK2-ntXP3ehQxsaHcQU8Z_nsogId7Pme5iJRnoHWEnWtbwz5DLSXL3ZZNnRdrHM9MdI7QSDz9mojKDCaMpGN9sG7Xl-tGdBp1XzXuUOzG8S03mtZ1IgVR1uiBL2N6oohHIAunk8DIAmNWI-zgycTgzUGU7mvPkKH43qO8Ua1-13tCUBKKa8VxcotZ67Mxm1QAvBGoDnTKwWMwghLzs6d6WViXQg6eWlJcpBA"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_rs384

Parses a claims object from a JWT string encoded with RS384. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The RS384 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_rs384("""-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs/ibN8r68pLMR6gRzg4S
8v8l6Q7yi8qURjkEbcNeM1rkokC7xh0I4JVTwxYSVv/JIW8qJdyspl5NIfuAVi32
WfKvSAs+NIs+DMsNPYw3yuQals4AX8hith1YDvYpr8SD44jxhz/DR9lYKZFGhXGB
+7NqQ7vpTWp3BceLYocazWJgusZt7CgecIq57ycM5hjM93BvlrUJ8nQ1a46wfL/8
Cy4P0et70hzZrsjjN41KFhKY0iUwlyU41yEiDHvHDDsTMBxAZosWjSREGfJL6Mfp
XOInTHs/Gg6DZMkbxjQu6L06EdJ+Q/NwglJdAXM7Zo9rNELqRig6DdvG5JesdMsO
+QIDAQAB
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJSUzM4NCIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.orcXYBcjVE5DU7mvq4KKWFfNdXR4nEY_xupzWoETRpYmQZIozlZnM_nHxEk2dySvpXlAzVm7kgOPK2RFtGlOVaNRIa3x-pMMr-bhZTno4L8Hl4sYxOks3bWtjK7wql4uqUbqThSJB12psAXw2-S-I_FMngOPGIn4jDT9b802ottJSvTpXcy0-eKTjrV2PSkRRu-EYJh0CJZW55MNhqlt6kCGhAXfbhNazN3ASX-dmpd_JixyBKphrngr_zRA-FCn_Xf3QQDA-5INopb4Yp5QiJ7UxVqQEKI80X_JvJqz9WE1qiAw8pq5-xTen1t7zTP-HT1NbbD3kltcNa3G8acmNg"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_rs512

Parses a claims object from a JWT string encoded with RS512. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The RS512 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_rs512("""-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs/ibN8r68pLMR6gRzg4S
8v8l6Q7yi8qURjkEbcNeM1rkokC7xh0I4JVTwxYSVv/JIW8qJdyspl5NIfuAVi32
WfKvSAs+NIs+DMsNPYw3yuQals4AX8hith1YDvYpr8SD44jxhz/DR9lYKZFGhXGB
+7NqQ7vpTWp3BceLYocazWJgusZt7CgecIq57ycM5hjM93BvlrUJ8nQ1a46wfL/8
Cy4P0et70hzZrsjjN41KFhKY0iUwlyU41yEiDHvHDDsTMBxAZosWjSREGfJL6Mfp
XOInTHs/Gg6DZMkbxjQu6L06EdJ+Q/NwglJdAXM7Zo9rNELqRig6DdvG5JesdMsO
+QIDAQAB
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.rsMp_X5HMrUqKnZJIxo27aAoscovRA6SSQYR9rq7pifIj0YHXxMyNyOBDGnvVALHKTi25VUGHpfNUW0VVMmae0A4t_ObNU6hVZHguWvetKZZq4FZpW1lgWHCMqgPGwT5_uOqwYCH6r8tJuZT3pqXeL0CY4putb1AN2w6CVp620nh3l8d3XWb4jaifycd_4CEVCqHuWDmohfug4VhmoVKlIXZkYoAQowgHlozATDssBSWdYtv107Wd2AzEoiXPu6e3pflsuXULlyqQnS4ELEKPYThFLafh1NqvZDPddqozcPZ-iODBW-xf3A4DYDdivnMYLrh73AZOGHexxu8ay6nDA"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

sign_jwt_es256

Hash and sign an object representing JSON Web Token (JWT) claims using ES256.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_es256("""-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_es256(signing_secret: """-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_es384

Hash and sign an object representing JSON Web Token (JWT) claims using ES384.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_es384("""-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_es384(signing_secret: """-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_es512

Hash and sign an object representing JSON Web Token (JWT) claims using ES512.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_es512("""-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_es512(signing_secret: """-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_hs256

Hash and sign an object representing JSON Web Token (JWT) claims using HS256.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_hs256("""dont-tell-anyone""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_hs256(signing_secret: """dont-tell-anyone""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_hs384

Hash and sign an object representing JSON Web Token (JWT) claims using HS384.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_hs384("""dont-tell-anyone""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_hs384(signing_secret: """dont-tell-anyone""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_hs512

Hash and sign an object representing JSON Web Token (JWT) claims using HS512.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_hs512("""dont-tell-anyone""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_hs512(signing_secret: """dont-tell-anyone""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_rs256

Hash and sign an object representing JSON Web Token (JWT) claims using RS256.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_rs256("""-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_rs256(signing_secret: """-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_rs384

Hash and sign an object representing JSON Web Token (JWT) claims using RS384.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_rs384("""-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_rs384(signing_secret: """-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_rs512

Hash and sign an object representing JSON Web Token (JWT) claims using RS512.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_rs512("""-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}
root.signed = this.claims.sign_jwt_rs512(signing_secret: """-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

Number manipulation

abs

Returns the absolute value of an int64 or float64 number. As a special case, when an integer is provided that is the minimum value it is converted to the maximum value.

Examples

root.outs = this.ins.map_each(ele -> ele.abs())

# In:  {"ins":[9,-18,1.23,-4.56]}
# Out: {"outs":[9,18,1.23,4.56]}

bitwise_and

Performs a bitwise AND operation between the integer and the specified value.

Parameters

Name Type Description

value

integer

The value to AND with

Examples

root.new_value = this.value.bitwise_and(6)

# In:  {"value":12}
# Out: {"new_value":4}
root.masked = this.flags.bitwise_and(15)

# In:  {"flags":127}
# Out: {"masked":15}

bitwise_or

Performs a bitwise OR operation between the integer and the specified value.

Parameters

Name Type Description

value

integer

The value to OR with

Examples

root.new_value = this.value.bitwise_or(6)

# In:  {"value":12}
# Out: {"new_value":14}
root.combined = this.flags.bitwise_or(8)

# In:  {"flags":4}
# Out: {"combined":12}

bitwise_xor

Performs a bitwise XOR (exclusive OR) operation between the integer and the specified value.

Parameters

Name Type Description

value

integer

The value to XOR with

Examples

root.new_value = this.value.bitwise_xor(6)

# In:  {"value":12}
# Out: {"new_value":10}
root.toggled = this.flags.bitwise_xor(5)

# In:  {"flags":3}
# Out: {"toggled":6}

ceil

Rounds a number up to the nearest integer. Returns an integer if the result fits in 64-bit, otherwise returns a float.

Examples

root.new_value = this.value.ceil()

# In:  {"value":5.3}
# Out: {"new_value":6}

# In:  {"value":-5.9}
# Out: {"new_value":-5}
root.result = this.price.ceil()

# In:  {"price":19.99}
# Out: {"result":20}

cos

Calculates the cosine of a given angle specified in radians.

Examples

root.new_value = (this.value * (pi() / 180)).cos()

# In:  {"value":45}
# Out: {"new_value":0.7071067811865476}

# In:  {"value":0}
# Out: {"new_value":1}

# In:  {"value":180}
# Out: {"new_value":-1}

float32

Converts a numerical type into a 32-bit floating point number, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 32-bit floating point number. Please refer to the strconv.ParseFloat documentation for details regarding the supported formats.

Examples

root.out = this.in.float32()

# In:  {"in":"6.674282313423543523453425345e-11"}
# Out: {"out":6.674283e-11}

float64

Converts a numerical type into a 64-bit floating point number, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 64-bit floating point number. Please refer to the strconv.ParseFloat documentation for details regarding the supported formats.

Examples

root.out = this.in.float64()

# In:  {"in":"6.674282313423543523453425345e-11"}
# Out: {"out":6.674282313423544e-11}

floor

Rounds a number down to the nearest integer. Returns an integer if the result fits in 64-bit, otherwise returns a float.

Examples

root.new_value = this.value.floor()

# In:  {"value":5.7}
# Out: {"new_value":5}

# In:  {"value":-3.2}
# Out: {"new_value":-4}
root.whole_seconds = this.duration_seconds.floor()

# In:  {"duration_seconds":12.345}
# Out: {"whole_seconds":12}

int16

Converts a numerical type into a 16-bit signed integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 16-bit signed integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.int16()
root.b = this.b.round().int16()
root.c = this.c.int16()
root.d = this.d.int16().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":-12}
root = this.int16()

# In:  "0xDE"
# Out: 222

int32

Converts a numerical type into a 32-bit signed integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 32-bit signed integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.int32()
root.b = this.b.round().int32()
root.c = this.c.int32()
root.d = this.d.int32().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":-12}
root = this.int32()

# In:  "0xDEAD"
# Out: 57005

int64

Converts a numerical type into a 64-bit signed integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 64-bit signed integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.int64()
root.b = this.b.round().int64()
root.c = this.c.int64()
root.d = this.d.int64().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":-12}
root = this.int64()

# In:  "0xDEADBEEF"
# Out: 3735928559

int8

Converts a numerical type into a 8-bit signed integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 8-bit signed integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.int8()
root.b = this.b.round().int8()
root.c = this.c.int8()
root.d = this.d.int8().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":-12}
root = this.int8()

# In:  "0xD"
# Out: 13

log

Calculates the natural logarithm (base e) of a number.

Examples

root.new_value = this.value.log().round()

# In:  {"value":1}
# Out: {"new_value":0}

# In:  {"value":2.7183}
# Out: {"new_value":1}
root.ln_result = this.number.log()

# In:  {"number":10}
# Out: {"ln_result":2.302585092994046}

log10

Calculates the base-10 logarithm of a number.

Examples

root.new_value = this.value.log10()

# In:  {"value":100}
# Out: {"new_value":2}

# In:  {"value":1000}
# Out: {"new_value":3}
root.log_value = this.magnitude.log10()

# In:  {"magnitude":10000}
# Out: {"log_value":4}

max

Returns the largest number from an array. All elements must be numbers and the array cannot be empty.

Examples

root.biggest = this.values.max()

# In:  {"values":[0,3,2.5,7,5]}
# Out: {"biggest":7}
root.highest_temp = this.temperatures.max()

# In:  {"temperatures":[20.5,22.1,19.8,23.4]}
# Out: {"highest_temp":23.4}

min

Returns the smallest number from an array. All elements must be numbers and the array cannot be empty.

Examples

root.smallest = this.values.min()

# In:  {"values":[0,3,-2.5,7,5]}
# Out: {"smallest":-2.5}
root.lowest_temp = this.temperatures.min()

# In:  {"temperatures":[20.5,22.1,19.8,23.4]}
# Out: {"lowest_temp":19.8}

pow

Returns the number raised to the specified exponent.

Parameters

Name Type Description

exponent

float

The exponent you want to raise to the power of.

Examples

root.new_value = this.value * 10.pow(-2)

# In:  {"value":2}
# Out: {"new_value":0.02}
root.new_value = this.value.pow(-2)

# In:  {"value":2}
# Out: {"new_value":0.25}

round

Rounds a number to the nearest integer. Values at .5 round away from zero. Returns an integer if the result fits in 64-bit, otherwise returns a float.

Examples

root.new_value = this.value.round()

# In:  {"value":5.3}
# Out: {"new_value":5}

# In:  {"value":5.9}
# Out: {"new_value":6}
root.rounded = this.score.round()

# In:  {"score":87.5}
# Out: {"rounded":88}

sin

Calculates the sine of a given angle specified in radians.

Examples

root.new_value = (this.value * (pi() / 180)).sin()

# In:  {"value":45}
# Out: {"new_value":0.7071067811865475}

# In:  {"value":0}
# Out: {"new_value":0}

# In:  {"value":90}
# Out: {"new_value":1}

tan

Calculates the tangent of a given angle specified in radians.

Examples

root.new_value = "%f".format((this.value * (pi() / 180)).tan())

# In:  {"value":0}
# Out: {"new_value":"0.000000"}

# In:  {"value":45}
# Out: {"new_value":"1.000000"}

# In:  {"value":180}
# Out: {"new_value":"-0.000000"}

uint16

Converts a numerical type into a 16-bit unsigned integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 16-bit unsigned integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.uint16()
root.b = this.b.round().uint16()
root.c = this.c.uint16()
root.d = this.d.uint16().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":0}
root = this.uint16()

# In:  "0xDE"
# Out: 222

uint32

Converts a numerical type into a 32-bit unsigned integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 32-bit unsigned integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.uint32()
root.b = this.b.round().uint32()
root.c = this.c.uint32()
root.d = this.d.uint32().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":0}
root = this.uint32()

# In:  "0xDEAD"
# Out: 57005

uint64

Converts a numerical type into a 64-bit unsigned integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 64-bit unsigned integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.uint64()
root.b = this.b.round().uint64()
root.c = this.c.uint64()
root.d = this.d.uint64().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":0}
root = this.uint64()

# In:  "0xDEADBEEF"
# Out: 3735928559

uint8

Converts a numerical type into a 8-bit unsigned integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 8-bit unsigned integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.uint8()
root.b = this.b.round().uint8()
root.c = this.c.uint8()
root.d = this.d.uint8().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":0}
root = this.uint8()

# In:  "0xD"
# Out: 13

Object & array manipulation

all

Tests whether all elements in an array satisfy a condition. Returns true only if the query evaluates to true for every element. Returns false for empty arrays.

Parameters

Name Type Description

test

query expression

A test query to apply to each element.

Examples

root.all_over_21 = this.patrons.all(patron -> patron.age >= 21)

# In:  {"patrons":[{"id":"1","age":18},{"id":"2","age":23}]}
# Out: {"all_over_21":false}

# In:  {"patrons":[{"id":"1","age":45},{"id":"2","age":23}]}
# Out: {"all_over_21":true}
root.all_positive = this.values.all(v -> v > 0)

# In:  {"values":[1,2,3,4,5]}
# Out: {"all_positive":true}

# In:  {"values":[1,-2,3,4,5]}
# Out: {"all_positive":false}

any

Tests whether at least one element in an array satisfies a condition. Returns true if the query evaluates to true for any element. Returns false for empty arrays.

Parameters

Name Type Description

test

query expression

A test query to apply to each element.

Examples

root.any_over_21 = this.patrons.any(patron -> patron.age >= 21)

# In:  {"patrons":[{"id":"1","age":18},{"id":"2","age":23}]}
# Out: {"any_over_21":true}

# In:  {"patrons":[{"id":"1","age":10},{"id":"2","age":12}]}
# Out: {"any_over_21":false}
root.has_errors = this.results.any(r -> r.status == "error")

# In:  {"results":[{"status":"ok"},{"status":"error"},{"status":"ok"}]}
# Out: {"has_errors":true}

# In:  {"results":[{"status":"ok"},{"status":"ok"}]}
# Out: {"has_errors":false}

append

Adds one or more elements to the end of an array and returns the new array. The original array is not modified.

Parameters

This method accepts any number of arguments.

Examples

root.foo = this.foo.append("and", "this")

# In:  {"foo":["bar","baz"]}
# Out: {"foo":["bar","baz","and","this"]}
root.combined = this.items.append(this.new_item)

# In:  {"items":["apple","banana"],"new_item":"orange"}
# Out: {"combined":["apple","banana","orange"]}

assign

Merges two objects or arrays with override behavior. For objects, source values replace destination values on key conflicts. Arrays are concatenated. To preserve both values on conflict, use the merge method instead.

Parameters

Name Type Description

with

unknown

A value to merge the target value with.

Examples

root = this.foo.assign(this.bar)

# In:  {"foo":{"first_name":"fooer","likes":"bars"},"bar":{"second_name":"barer","likes":"foos"}}
# Out: {"first_name":"fooer","likes":"foos","second_name":"barer"}

Override defaults with user settings:

root.config = this.defaults.assign(this.user_settings)

# In:  {"defaults":{"timeout":30,"retries":3},"user_settings":{"timeout":60}}
# Out: {"config":{"retries":3,"timeout":60}}

collapse

Flattens a nested structure into a single-level object with dot-notation keys representing the full path to each value. Empty arrays and objects are excluded by default.

Parameters

Name Type Description

include_empty (optional)

bool

Whether to include empty objects and arrays in the resulting object. Default: false.

Examples

root.result = this.collapse()

# In:  {"foo":[{"bar":"1"},{"bar":{}},{"bar":"2"},{"bar":[]}]}
# Out: {"result":{"foo.0.bar":"1","foo.2.bar":"2"}}

Set include_empty to true to preserve empty objects and arrays in the output:

root.result = this.collapse(include_empty: true)

# In:  {"foo":[{"bar":"1"},{"bar":{}},{"bar":"2"},{"bar":[]}]}
# Out: {"result":{"foo.0.bar":"1","foo.1.bar":{},"foo.2.bar":"2","foo.3.bar":[]}}

concat

Concatenates an array value with one or more argument arrays.

Parameters

This method accepts any number of arguments.

Examples

root.foo = this.foo.concat(this.bar, this.baz)

# In:  {"foo":["a","b"],"bar":["c"],"baz":["d","e","f"]}
# Out: {"foo":["a","b","c","d","e","f"]}

contains

Checks whether an array contains an element matching the argument, or an object contains a value matching the argument, and returns a boolean result. Numerical comparisons are made irrespective of the representation type (float versus integer).

Parameters

Name Type Description

value

unknown

A value to test against elements of the target.

Examples

root.has_foo = this.thing.contains("foo")

# In:  {"thing":["this","foo","that"]}
# Out: {"has_foo":true}

# In:  {"thing":["this","bar","that"]}
# Out: {"has_foo":false}
root.has_bar = this.thing.contains(20)

# In:  {"thing":[10.3,20.0,"huh",3]}
# Out: {"has_bar":true}

# In:  {"thing":[2,3,40,67]}
# Out: {"has_bar":false}

diff

Compares the current value with another value and returns a detailed changelog describing all differences. The changelog contains operations (create, update, delete) with their paths and values, enabling you to track changes between data versions, implement audit logs, or synchronize data between systems.

Parameters

Name Type Description

other

unknown

The value to compare against the current value. Can be any structured data (object or array).

Examples

Compare two objects to track field changes:

root.changes = this.before.diff(this.after)

# In:  {"before":{"name":"Alice","age":30},"after":{"name":"Alice","age":31,"city":"NYC"}}
# Out: {"changes":[{"From":30,"Path":["age"],"To":31,"Type":"update"},{"From":null,"Path":["city"],"To":"NYC","Type":"create"}]}

Detect deletions in configuration changes:

root.changelog = this.old_config.diff(this.new_config)

# In:  {"old_config":{"debug":true,"timeout":30},"new_config":{"timeout":60}}
# Out: {"changelog":[{"From":true,"Path":["debug"],"To":null,"Type":"delete"},{"From":30,"Path":["timeout"],"To":60,"Type":"update"}]}

enumerated

Transforms an array into an array of objects with index and value fields, making it easy to access both the position and content of each element.

Examples

root.foo = this.foo.enumerated()

# In:  {"foo":["bar","baz"]}
# Out: {"foo":[{"index":0,"value":"bar"},{"index":1,"value":"baz"}]}

Useful for filtering by index position:

root.first_two = this.items.enumerated().filter(item -> item.index < 2).map_each(item -> item.value)

# In:  {"items":["a","b","c","d"]}
# Out: {"first_two":["a","b"]}

exists

Checks whether a field exists at the specified dot path within an object. Returns true if the field is present (even if null), false otherwise.

Parameters

Name Type Description

path

string

A dot path to a field.

Examples

root.result = this.foo.exists("bar.baz")

# In:  {"foo":{"bar":{"baz":"yep, I exist"}}}
# Out: {"result":true}

# In:  {"foo":{"bar":{}}}
# Out: {"result":false}

# In:  {"foo":{}}
# Out: {"result":false}

Also returns true for null values if the field exists:

root.has_field = this.data.exists("optional_field")

# In:  {"data":{"optional_field":null}}
# Out: {"has_field":true}

# In:  {"data":{}}
# Out: {"has_field":false}

explode

Expands a nested array or object field into multiple documents, distributing elements while preserving the surrounding structure. Useful for denormalizing data.

Parameters

Name Type Description

path

string

A dot path to a field to explode.

Examples

On arrays

When exploding an array, each element becomes a separate document with the array element replacing the original field:

root = this.explode("value")

# In:  {"id":1,"value":["foo","bar","baz"]}
# Out: [{"id":1,"value":"foo"},{"id":1,"value":"bar"},{"id":1,"value":"baz"}]
On objects

When exploding an object, the output keys match the nested object’s keys, with values being the full document where the target field is replaced by each nested value:

root = this.explode("value")

# In:  {"id":1,"value":{"foo":2,"bar":[3,4],"baz":{"bev":5}}}
# Out: {"bar":{"id":1,"value":[3,4]},"baz":{"id":1,"value":{"bev":5}},"foo":{"id":1,"value":2}}

filter

Returns a new array or object containing only elements that satisfy the provided condition. Elements for which the query returns true are kept, all others are removed.

Parameters

Name Type Description

test

query expression

A query to apply to each element, if this query resolves to any value other than a boolean true the element will be removed from the result.

Examples

root.new_nums = this.nums.filter(num -> num > 10)

# In:  {"nums":[3,11,4,17]}
# Out: {"new_nums":[11,17]}
On objects

When filtering objects, the query receives a context with key and value fields for each entry:

root.new_dict = this.dict.filter(item -> item.value.contains("foo"))

# In:  {"dict":{"first":"hello foo","second":"world","third":"this foo is great"}}
# Out: {"new_dict":{"first":"hello foo","third":"this foo is great"}}

find

This method is in beta. Its behavior might change in a future release.

Searches an array for a matching value and returns the index of the first occurrence. Returns -1 if no match is found. Numeric types are compared by value regardless of representation.

Parameters

Name Type Description

value

unknown

A value to find.

Examples

root.index = this.find("bar")

# In:  ["foo", "bar", "baz"]
# Out: {"index":1}
root.index = this.things.find(this.goal)

# In:  {"goal":"bar","things":["foo", "bar", "baz"]}
# Out: {"index":1}

find_all

This method is in beta. Its behavior might change in a future release.

Searches an array for all occurrences of a value and returns an array of matching indexes. Returns an empty array if no matches are found. Numeric types are compared by value regardless of representation.

Parameters

Name Type Description

value

unknown

A value to find.

Examples

root.index = this.find_all("bar")

# In:  ["foo", "bar", "baz", "bar"]
# Out: {"index":[1,3]}
root.indexes = this.things.find_all(this.goal)

# In:  {"goal":"bar","things":["foo", "bar", "baz", "bar", "buz"]}
# Out: {"indexes":[1,3]}

find_all_by

This method is in beta. Its behavior might change in a future release.

Searches an array for all elements that satisfy a condition and returns an array of their indexes. Returns an empty array if no elements match.

Parameters

Name Type Description

query

query expression

A query to execute for each element.

Examples

root.index = this.find_all_by(v -> v != "bar")

# In:  ["foo", "bar", "baz"]
# Out: {"index":[0,2]}

Find all indexes matching criteria:

root.error_indexes = this.logs.find_all_by(log -> log.level == "error")

# In:  {"logs":[{"level":"info"},{"level":"error"},{"level":"warn"},{"level":"error"}]}
# Out: {"error_indexes":[1,3]}

find_by

This method is in beta. Its behavior might change in a future release.

Searches an array for the first element that satisfies a condition and returns its index. Returns -1 if no element matches the query.

Parameters

Name Type Description

query

query expression

A query to execute for each element.

Examples

root.index = this.find_by(v -> v != "bar")

# In:  ["foo", "bar", "baz"]
# Out: {"index":0}

Find first object matching criteria:

root.first_adult = this.users.find_by(u -> u.age >= 18)

# In:  {"users":[{"name":"Alice","age":15},{"name":"Bob","age":22},{"name":"Carol","age":19}]}
# Out: {"first_adult":1}

flatten

Flattens an array by one level, expanding nested arrays into the parent array. Only the first level of nesting is removed.

Examples

root.result = this.flatten()

# In:  ["foo",["bar","baz"],"buz"]
# Out: {"result":["foo","bar","baz","buz"]}

Deeper nesting requires multiple flatten calls:

root.result = this.data.flatten()

# In:  {"data":["a",["b",["c","d"]],"e"]}
# Out: {"result":["a","b",["c","d"],"e"]}

fold

Reduces an array to a single value by iteratively applying a function. Also known as reduce or aggregate. The query receives an accumulator (tally) and current element (value) for each iteration.

Parameters

Name Type Description

initial

unknown

The initial value to start the fold with. For example, an empty object {}, a zero count 0, or an empty string "".

query

query expression

A query to apply for each element. The query is provided an object with two fields; tally containing the current tally, and value containing the value of the current element. The query should result in a new tally to be passed to the next element query.

Examples

Sum numbers in an array:

root.sum = this.foo.fold(0, item -> item.tally + item.value)

# In:  {"foo":[3,8,11]}
# Out: {"sum":22}

Concatenate strings:

root.result = this.foo.fold("", item -> "%v%v".format(item.tally, item.value))

# In:  {"foo":["hello ", "world"]}
# Out: {"result":"hello world"}

Merge an array of objects into a single object:

root.smoothie = this.fruits.fold({}, item -> item.tally.merge(item.value))

# In:  {"fruits":[{"apple":5},{"banana":3},{"orange":8}]}
# Out: {"smoothie":{"apple":5,"banana":3,"orange":8}}

get

Extract a field value, identified via a dot path, from an object.

Parameters

Name Type Description

path

string

A dot path identifying a field to obtain.

Examples

root.result = this.foo.get(this.target)

# In:  {"foo":{"bar":"from bar","baz":"from baz"},"target":"bar"}
# Out: {"result":"from bar"}

# In:  {"foo":{"bar":"from bar","baz":"from baz"},"target":"baz"}
# Out: {"result":"from baz"}

index

Extract an element from an array by an index. The index can be negative, and if so the element will be selected from the end counting backwards starting from -1. E.g. an index of -1 returns the last element, an index of -2 returns the element before the last, and so on.

Parameters

Name Type Description

index

integer

The index to obtain from an array.

Examples

root.last_name = this.names.index(-1)

# In:  {"names":["rachel","stevens"]}
# Out: {"last_name":"stevens"}

It is also possible to use this method on byte arrays, in which case the selected element will be returned as an integer:

root.last_byte = this.name.bytes().index(-1)

# In:  {"name":"foobar bazson"}
# Out: {"last_byte":110}

join

Concatenates an array of strings into a single string with an optional delimiter between elements. Use for building CSV strings, URLs, or combining text fragments.

Parameters

Name Type Description

delimiter (optional)

string

An optional delimiter to add between each string.

Examples

root.joined_words = this.words.join()
root.joined_numbers = this.numbers.map_each(this.string()).join(",")

# In:  {"words":["hello","world"],"numbers":[3,8,11]}
# Out: {"joined_numbers":"3,8,11","joined_words":"helloworld"}

json_path

Executes the given JSONPath expression on an object or array and returns the result. The JSONPath expression syntax can be found at https://goessner.net/articles/JsonPath/. For more complex logic, you can use Gval expressions (https://github.com/PaesslerAG/gval).

Parameters

Name Type Description

expression

string

The JSONPath expression to execute.

Examples

root.all_names = this.json_path("$..name")

# In:  {"name":"alice","foo":{"name":"bob"}}
# Out: {"all_names":["alice","bob"]}

# In:  {"thing":["this","bar",{"name":"alice"}]}
# Out: {"all_names":["alice"]}
root.text_objects = this.json_path("$.body[?(@.type=='text')]")

# In:  {"body":[{"type":"image","id":"foo"},{"type":"text","id":"bar"}]}
# Out: {"text_objects":[{"id":"bar","type":"text"}]}

json_schema

This method is in beta. Its behavior might change in a future release.

Checks a JSON schema against a value and returns the value if it matches or throws an error if it does not. To load the schema from a file, pass it the result of the file function, which isn’t available in Redpanda Cloud.

Parameters

Name Type Description

schema

string

The schema to check values against.

Examples

root = this.json_schema("""{
  "type":"object",
  "properties":{
    "foo":{
      "type":"string"
    }
  }
}""")

# In:  {"foo":"bar"}
# Out: {"foo":"bar"}

# In:  {"foo":5}
# Out: Error("failed assignment (line 1): field `this`: foo invalid type. expected: string, given: integer")

key_values

Converts an object into an array of key-value pair objects. Each element has a 'key' field and a 'value' field. Order is not guaranteed unless sorted.

Examples

root.foo_key_values = this.foo.key_values().sort_by(pair -> pair.key)

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_key_values":[{"key":"bar","value":1},{"key":"baz","value":2}]}

Filter object entries by value:

root.large_items = this.items.key_values().filter(pair -> pair.value > 15).map_each(pair -> pair.key)

# In:  {"items":{"a":5,"b":15,"c":20,"d":3}}
# Out: {"large_items":["c"]}

keys

Extracts all keys from an object and returns them as a sorted array.

Examples

root.foo_keys = this.foo.keys()

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_keys":["bar","baz"]}

Check if specific keys exist:

root.has_id = this.data.keys().contains("id")

# In:  {"data":{"id":123,"name":"test"}}
# Out: {"has_id":true}

length

Returns the size of an array (element count) or object (key count).

Examples

root.foo_len = this.foo.length()

# In:  {"foo":["first","second"]}
# Out: {"foo_len":2}

# In:  {"foo":{"first":"bar","second":"baz"}}
# Out: {"foo_len":2}

map_each

Applies a mapping query to each element of an array or each value in an object. Returns a new collection with the transformed values.

Parameters

Name Type Description

query

query expression

A query that will be used to map each element.

Examples

On arrays

Transforms each array element using a query. Return deleted() to remove an element, or the new value to replace it:

root.new_nums = this.nums.map_each(num -> if num < 10 {
  deleted()
} else {
  num - 10
})

# In:  {"nums":[3,11,4,17]}
# Out: {"new_nums":[1,7]}
On objects

Transforms each object value using a query. The query receives an object with 'key' and 'value' fields for each entry:

root.new_dict = this.dict.map_each(item -> item.value.uppercase())

# In:  {"dict":{"foo":"hello","bar":"world"}}
# Out: {"new_dict":{"bar":"WORLD","foo":"HELLO"}}

map_each_key

Transforms object keys using a query. The query receives each key as a string and must return a new string key. Use this to rename or transform keys while preserving values.

Parameters

Name Type Description

query

query expression

A query that will be used to map each key.

Examples

root.new_dict = this.dict.map_each_key(key -> key.uppercase())

# In:  {"dict":{"keya":"hello","keyb":"world"}}
# Out: {"new_dict":{"KEYA":"hello","KEYB":"world"}}

Conditionally transform keys:

root = this.map_each_key(key -> if key.contains("kafka") { "_" + key })

# In:  {"amqp_key":"foo","kafka_key":"bar","kafka_topic":"baz"}
# Out: {"_kafka_key":"bar","_kafka_topic":"baz","amqp_key":"foo"}

merge

Combines two objects or arrays. When merging objects, conflicting keys create arrays containing both values. Arrays are concatenated. For key override behavior instead, use the assign method.

Parameters

Name Type Description

with

unknown

A value to merge the target value with.

Examples

root = this.foo.merge(this.bar)

# In:  {"foo":{"first_name":"fooer","likes":"bars"},"bar":{"second_name":"barer","likes":"foos"}}
# Out: {"first_name":"fooer","likes":["bars","foos"],"second_name":"barer"}

Merge arrays:

root.combined = this.list1.merge(this.list2)

# In:  {"list1":["a","b"],"list2":["c","d"]}
# Out: {"combined":["a","b","c","d"]}

patch

Applies a changelog (created by the diff method) to the current value, transforming it according to the specified operations. This enables you to synchronize data, replay changes, or implement event sourcing patterns by applying recorded changes to reconstruct state.

Parameters

Name Type Description

changelog

unknown

The changelog array to apply. Should be in the format returned by the diff method, containing Type, Path, From, and To fields for each change.

Examples

Apply recorded changes to update an object:

root.updated = this.current.patch(this.changelog)

# In:  {"current":{"name":"Alice","age":30},"changelog":[{"Type":"update","Path":["age"],"From":30,"To":31},{"Type":"create","Path":["city"],"From":null,"To":"NYC"}]}
# Out: {"updated":{"age":31,"city":"NYC","name":"Alice"}}

Restore previous state by applying inverse changes:

root.restored = this.modified.patch(this.reverse_changelog)

# In:  {"modified":{"timeout":60},"reverse_changelog":[{"Type":"create","Path":["debug"],"From":null,"To":true},{"Type":"update","Path":["timeout"],"From":60,"To":30}]}
# Out: {"restored":{"debug":true,"timeout":30}}

slice

Extract a slice from an array by specifying two indices, a low and high bound, which selects a half-open range that includes the first element, but excludes the last one. If the second index is omitted then it defaults to the length of the input sequence.

Parameters

Name Type Description

low

integer

The low bound, which is the first element of the selection, or if negative selects from the end.

high (optional)

integer

An optional high bound.

Examples

root.beginning = this.value.slice(0, 2)
root.end = this.value.slice(4)

# In:  {"value":["foo","bar","baz","buz","bev"]}
# Out: {"beginning":["foo","bar"],"end":["bev"]}

A negative low index can be used, indicating an offset from the end of the sequence. If the low index is greater than the length of the sequence then an empty result is returned:

root.last_chunk = this.value.slice(-2)
root.the_rest = this.value.slice(0, -2)

# In:  {"value":["foo","bar","baz","buz","bev"]}
# Out: {"last_chunk":["buz","bev"],"the_rest":["foo","bar","baz"]}

sort

Sorts an array in ascending order. Works with strings and numbers. For custom sorting logic, provide a comparison query that receives 'left' and 'right' elements.

Parameters

Name Type Description

compare (optional)

query expression

An optional query that should explicitly compare elements left and right and provide a boolean result.

Examples

root.sorted = this.foo.sort()

# In:  {"foo":["bbb","ccc","aaa"]}
# Out: {"sorted":["aaa","bbb","ccc"]}

Custom comparison for complex objects - return true if left < right:

root.sorted = this.foo.sort(item -> item.left.v < item.right.v)

# In:  {"foo":[{"id":"foo","v":"bbb"},{"id":"bar","v":"ccc"},{"id":"baz","v":"aaa"}]}
# Out: {"sorted":[{"id":"baz","v":"aaa"},{"id":"foo","v":"bbb"},{"id":"bar","v":"ccc"}]}

sort_by

Sorts an array by a value extracted from each element using a query. The extracted values determine sort order and must all be strings or numbers.

Parameters

Name Type Description

query

query expression

A query to apply to each element that yields a value used for sorting.

Examples

root.sorted = this.foo.sort_by(ele -> ele.id)

# In:  {"foo":[{"id":"bbb","message":"bar"},{"id":"aaa","message":"foo"},{"id":"ccc","message":"baz"}]}
# Out: {"sorted":[{"id":"aaa","message":"foo"},{"id":"bbb","message":"bar"},{"id":"ccc","message":"baz"}]}

Sort by numeric field:

root.sorted = this.items.sort_by(item -> item.priority)

# In:  {"items":[{"name":"low","priority":3},{"name":"high","priority":1},{"name":"med","priority":2}]}
# Out: {"sorted":[{"name":"high","priority":1},{"name":"med","priority":2},{"name":"low","priority":3}]}

squash

Squashes an array of objects into a single object, where key collisions result in the values being merged (following similar rules as the .merge() method).

Examples

root.locations = this.locations.map_each(loc -> {loc.state: [loc.name]}).squash()

# In:  {"locations":[{"name":"Seattle","state":"WA"},{"name":"New York","state":"NY"},{"name":"Bellevue","state":"WA"},{"name":"Olympia","state":"WA"}]}
# Out: {"locations":{"NY":["New York"],"WA":["Seattle","Bellevue","Olympia"]}}

sum

Calculates the sum of all numeric values in an array. Non-numeric values cause an error.

Examples

root.sum = this.foo.sum()

# In:  {"foo":[3,8,4]}
# Out: {"sum":15}

Works with decimals:

root.total = this.prices.sum()

# In:  {"prices":[10.5,20.25,5.00]}
# Out: {"total":35.75}

unique

Removes duplicate values from an array, keeping the first occurrence of each unique value. Strings and numbers are treated as distinct types ("5" differs from 5).

Parameters

Name Type Description

emit (optional)

query expression

An optional query that can be used in order to yield a value for each element to determine uniqueness.

Examples

root.uniques = this.foo.unique()

# In:  {"foo":["a","b","a","c"]}
# Out: {"uniques":["a","b","c"]}

Use a query to determine uniqueness by a field:

root.unique_users = this.users.unique(u -> u.id)

# In:  {"users":[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"},{"id":1,"name":"Alice Duplicate"}]}
# Out: {"unique_users":[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"}]}

values

Extracts all values from an object and returns them as an array. Order is not guaranteed unless the result is sorted.

Examples

root.foo_vals = this.foo.values().sort()

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_vals":[1,2]}

Find max value in object:

root.max = this.scores.values().sort().index(-1)

# In:  {"scores":{"player1":85,"player2":92,"player3":78}}
# Out: {"max":92}

with

Returns an object where all but one or more field path arguments are removed. Each path specifies a specific field to be retained from the input object, allowing for nested fields.

If a key within a nested path does not exist then it is ignored.

Parameters

This method accepts any number of arguments.

Examples

root = this.with("inner.a","inner.c","d")

# In:  {"inner":{"a":"first","b":"second","c":"third"},"d":"fourth","e":"fifth"}
# Out: {"d":"fourth","inner":{"a":"first","c":"third"}}

without

Removes specified fields from an object using dot-notation paths. Returns a new object with the fields removed. Non-existent paths are safely ignored.

Parameters

This method accepts any number of arguments.

Examples

root = this.without("inner.a","inner.c","d")

# In:  {"inner":{"a":"first","b":"second","c":"third"},"d":"fourth","e":"fifth"}
# Out: {"e":"fifth","inner":{"b":"second"}}

Remove sensitive fields:

root = this.without("password","ssn","creditCard")

# In:  {"username":"alice","password":"secret","email":"alice@example.com","ssn":"123-45-6789"}
# Out: {"email":"alice@example.com","username":"alice"}

zip

Zip an array value with one or more argument arrays. Each array must match in length.

Parameters

This method accepts any number of arguments.

Examples

root.foo = this.foo.zip(this.bar, this.baz)

# In:  {"foo":["a","b","c"],"bar":[1,2,3],"baz":[4,5,6]}
# Out: {"foo":[["a",1,4],["b",2,5],["c",3,6]]}

Parsing

bloblang

This method is in beta. Its behavior might change in a future release.

Executes an argument Bloblang mapping on the target. This method can be used in order to execute dynamic mappings. Imports and functions that interact with the environment, such as file and env, or that access message information directly, such as content or json, are not enabled for dynamic Bloblang mappings.

Parameters

Name Type Description

mapping

string

The mapping to execute.

Examples

root.body = this.body.bloblang(this.mapping)

# In:  {"body":{"foo":"hello world"},"mapping":"root.foo = this.foo.uppercase()"}
# Out: {"body":{"foo":"HELLO WORLD"}}

# In:  {"body":{"foo":"hello world 2"},"mapping":"root.foo = this.foo.capitalize()"}
# Out: {"body":{"foo":"Hello World 2"}}

format_json

This method is in beta. Its behavior might change in a future release.

Serializes a target value into a pretty-printed JSON byte array (with 4 space indentation by default).

Parameters

Name Type Description

indent (optional)

string

Indentation string. Each element in a JSON object or array will begin on a new, indented line followed by one or more copies of indent according to the indentation nesting. Default: " ".

no_indent (optional)

bool

Disable indentation. Default: false.

escape_html (optional)

bool

Escape problematic HTML characters. Default: true.

Examples

root = this.doc.format_json()

# In:  {"doc":{"foo":"bar"}}
# Out: {
    "foo": "bar"
}

Pass a string to the indent parameter in order to customise the indentation:

root = this.format_json("  ")

# In:  {"doc":{"foo":"bar"}}
# Out: {
  "doc": {
    "foo": "bar"
  }
}

Use the .string() method in order to coerce the result into a string:

root.doc = this.doc.format_json().string()

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":"{\n    \"foo\": \"bar\"\n}"}

Set the no_indent parameter to true to disable indentation. The result is equivalent to calling bytes():

root = this.doc.format_json(no_indent: true)

# In:  {"doc":{"foo":"bar"}}
# Out: {"foo":"bar"}

Escapes problematic HTML characters:

root = this.doc.format_json()

# In:  {"doc":{"email":"foo&bar@benthos.dev","name":"foo>bar"}}
# Out: {
    "email": "foo\u0026bar@benthos.dev",
    "name": "foo\u003ebar"
}

Set the escape_html parameter to false to disable escaping of problematic HTML characters:

root = this.doc.format_json(escape_html: false)

# In:  {"doc":{"email":"foo&bar@benthos.dev","name":"foo>bar"}}
# Out: {
    "email": "foo&bar@benthos.dev",
    "name": "foo>bar"
}

format_msgpack

Serializes structured data into MessagePack binary format. MessagePack is a compact binary serialization that is faster and more space-efficient than JSON, making it ideal for network transmission and storage of structured data. Returns a byte array that can be further encoded as needed.

Examples

Serialize object to MessagePack and encode as hex for transmission:

root = this.format_msgpack().encode("hex")

# In:  {"foo":"bar"}
# Out: 81a3666f6fa3626172

Serialize data to MessagePack and base64 encode for embedding in JSON:

root.msgpack_payload = this.data.format_msgpack().encode("base64")

# In:  {"data":{"foo":"bar"}}
# Out: {"msgpack_payload":"gaNmb2+jYmFy"}

format_xml

Serializes an object into an XML document. Converts structured data to XML format with support for attributes (prefixed with hyphen), custom indentation, and configurable root element. Returns XML as a byte array.

Parameters

Name Type Description

indent (optional)

string

String to use for each level of indentation (default is 4 spaces). Each nested XML element will be indented by this string. Default: " ".

no_indent (optional)

bool

Disable indentation and newlines to produce compact XML on a single line. Default: false.

root_tag (optional)

string

Custom name for the root XML element. By default, the root element name is derived from the first key in the object.

Examples

Serialize object to pretty-printed XML with default indentation:

root = this.format_xml()

# In:  {"foo":{"bar":{"baz":"foo bar baz"}}}
# Out: <foo>
    <bar>
        <baz>foo bar baz</baz>
    </bar>
</foo>

Create compact XML without indentation for smaller message size:

root = this.format_xml(no_indent: true)

# In:  {"foo":{"bar":{"baz":"foo bar baz"}}}
# Out: <foo><bar><baz>foo bar baz</baz></bar></foo>

format_yaml

Serializes a target value into a YAML byte array.

Examples

root = this.doc.format_yaml()

# In:  {"doc":{"foo":"bar"}}
# Out: foo: bar

Use the .string() method in order to coerce the result into a string:

root.doc = this.doc.format_yaml().string()

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":"foo: bar\n"}

infer_schema

Attempt to infer the schema of a given value. The resulting schema can then be used as an input to schema conversion and enforcement methods.

parse_csv

Attempts to parse a string into an array of objects by following the CSV format described in RFC 4180.

Parameters

Name Type Description

parse_header_row (optional)

bool

Whether to reference the first row as a header row. If set to true the output structure for messages will be an object where field keys are determined by the header row. Otherwise, the output will be an array of row arrays. Default: true.

delimiter (optional)

string

The delimiter to use for splitting values in each record. It must be a single character. Default: ",".

lazy_quotes (optional)

bool

If set to true, a quote may appear in an unquoted field and a non-doubled quote may appear in a quoted field. Default: false.

Examples

Parses CSV data with a header row:

root.orders = this.orders.parse_csv()

# In:  {"orders":"foo,bar\nfoo 1,bar 1\nfoo 2,bar 2"}
# Out: {"orders":[{"bar":"bar 1","foo":"foo 1"},{"bar":"bar 2","foo":"foo 2"}]}

Parses CSV data without a header row:

root.orders = this.orders.parse_csv(false)

# In:  {"orders":"foo 1,bar 1\nfoo 2,bar 2"}
# Out: {"orders":[["foo 1","bar 1"],["foo 2","bar 2"]]}

Parses CSV data delimited by dots:

root.orders = this.orders.parse_csv(delimiter:".")

# In:  {"orders":"foo.bar\nfoo 1.bar 1\nfoo 2.bar 2"}
# Out: {"orders":[{"bar":"bar 1","foo":"foo 1"},{"bar":"bar 2","foo":"foo 2"}]}

Parses CSV data containing a quote in an unquoted field:

root.orders = this.orders.parse_csv(lazy_quotes:true)

# In:  {"orders":"foo,bar\nfoo 1,bar 1\nfoo\" \"2,bar\" \"2"}
# Out: {"orders":[{"bar":"bar 1","foo":"foo 1"},{"bar":"bar\" \"2","foo":"foo\" \"2"}]}

parse_form_url_encoded

Attempts to parse a url-encoded query string (from an x-www-form-urlencoded request body) and returns a structured result.

Examples

root.values = this.body.parse_form_url_encoded()

# In:  {"body":"noise=meow&animal=cat&fur=orange&fur=fluffy"}
# Out: {"values":{"animal":"cat","fur":["orange","fluffy"],"noise":"meow"}}

parse_json

Attempts to parse a string as a JSON document and returns the result.

Parameters

Name Type Description

use_number (optional)

bool

An optional flag that when set makes parsing numbers as json.Number instead of the default float64.

Examples

root.doc = this.doc.parse_json()

# In:  {"doc":"{\"foo\":\"bar\"}"}
# Out: {"doc":{"foo":"bar"}}
root.doc = this.doc.parse_json(use_number: true)

# In:  {"doc":"{\"foo\":\"11380878173205700000000000000000000000000000000\"}"}
# Out: {"doc":{"foo":"11380878173205700000000000000000000000000000000"}}

parse_logfmt

Attempts to parse a logfmt encoded string into an object. A logfmt string contains key=value pairs separated by spaces, where values can optionally be quoted.

Examples

root = this.msg.parse_logfmt()

# In:  {"msg":"level=info msg=\"hello world\" dur=1.5s"}
# Out: {"dur":"1.5s","level":"info","msg":"hello world"}

parse_msgpack

Parses MessagePack binary data into a structured object. MessagePack is an efficient binary serialization format that is more compact than JSON while maintaining similar data structures. Commonly used for high-performance APIs and data interchange between microservices.

Examples

Parse MessagePack data from hex-encoded content:

root = content().decode("hex").parse_msgpack()

# In:  81a3666f6fa3626172
# Out: {"foo":"bar"}

Parse MessagePack from base64-encoded field:

root.decoded = this.msgpack_data.decode("base64").parse_msgpack()

# In:  {"msgpack_data":"gaNmb2+jYmFy"}
# Out: {"decoded":{"foo":"bar"}}

parse_parquet

Parses Apache Parquet binary data into an array of objects. Parquet is a columnar storage format optimized for analytics, commonly used with big data systems like Apache Spark, Hive, and cloud data warehouses. Each row in the Parquet file becomes an object in the output array.

Parameters

Name Type Description

byte_array_as_string (optional)

bool

Deprecated: This parameter is no longer used. Default: false.

Examples

Parse Parquet file data into structured objects:

root.records = content().parse_parquet()

Process Parquet data from a field and extract specific columns:

root.users = this.parquet_data.parse_parquet().map_each(row -> {"name": row.name, "email": row.email})

parse_url

Attempts to parse a URL from a string value, returning a structured result that describes the various facets of the URL. The fields returned within the structured result roughly follow https://pkg.go.dev/net/url#URL, and may be expanded in future in order to present more information.

Examples

root.foo_url = this.foo_url.parse_url()

# In:  {"foo_url":"https://docs.redpanda.com/redpanda-connect/guides/bloblang/about/"}
# Out: {"foo_url":{"fragment":"","host":"docs.redpanda.com","opaque":"","path":"/redpanda-connect/guides/bloblang/about/","raw_fragment":"","raw_path":"","raw_query":"","scheme":"https"}}
root.username = this.url.parse_url().user.name | "unknown"

# In:  {"url":"amqp://foo:bar@127.0.0.1:5672/"}
# Out: {"username":"foo"}

# In:  {"url":"redis://localhost:6379"}
# Out: {"username":"unknown"}

parse_xml

Parses an XML document into a structured object. Converts XML elements to JSON-like objects following these rules:

  • Element attributes are prefixed with a hyphen (e.g., -id for an id attribute)

  • Elements with both attributes and text content store the text in a #text field

  • Repeated elements become arrays

  • XML comments, directives, and processing instructions are ignored

  • Optionally cast numeric and boolean strings to their proper types.

Parameters

Name Type Description

cast (optional)

bool

Whether to automatically cast numeric and boolean string values to their proper types. When false, all values remain as strings. Default: false.

Examples

Parse XML document into object structure:

root.doc = this.doc.parse_xml()

# In:  {"doc":"<root><title>This is a title</title><content>This is some content</content></root>"}
# Out: {"doc":{"root":{"content":"This is some content","title":"This is a title"}}}

Parse XML with type casting enabled to convert strings to numbers and booleans:

root.doc = this.doc.parse_xml(cast: true)

# In:  {"doc":"<root><title>This is a title</title><number id=\"99\">123</number><bool>True</bool></root>"}
# Out: {"doc":{"root":{"bool":true,"number":{"#text":123,"-id":99},"title":"This is a title"}}}

parse_yaml

Attempts to parse a string as a single YAML document and returns the result.

Examples

root.doc = this.doc.parse_yaml()

# In:  {"doc":"foo: bar"}
# Out: {"doc":{"foo":"bar"}}

Regular expressions

re_find_all

Finds all matches of a regular expression in a string and returns them as an array. Use for extracting multiple patterns or validating repeating structures.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_find_all("a.")

# In:  {"value":"paranormal"}
# Out: {"matches":["ar","an","al"]}
root.numbers = this.text.re_find_all("[0-9]+")

# In:  {"text":"I have 2 apples and 15 oranges"}
# Out: {"numbers":["2","15"]}

re_find_all_object

Finds all regex matches and returns an array of objects with named capture groups as keys. Each object represents one match with its captured groups. Use for parsing multiple structured records from text.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_find_all_object("a(?P<foo>x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":[{"0":"axxb","foo":"xx"},{"0":"ab","foo":""}]}
root.matches = this.value.re_find_all_object("(?m)(?P<key>\\w+):\\s+(?P<value>\\w+)$")

# In:  {"value":"option1: value1\noption2: value2\noption3: value3"}
# Out: {"matches":[{"0":"option1: value1","key":"option1","value":"value1"},{"0":"option2: value2","key":"option2","value":"value2"},{"0":"option3: value3","key":"option3","value":"value3"}]}

re_find_all_submatch

Finds all regex matches and their capture groups, returning an array of arrays where each inner array contains the full match and captured subgroups. Use for extracting structured data with capture groups.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_find_all_submatch("a(x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":[["axxb","xx"],["ab",""]]}
root.emails = this.text.re_find_all_submatch("(\\w+)@(\\w+\\.\\w+)")

# In:  {"text":"Contact: alice@example.com or bob@test.org"}
# Out: {"emails":[["alice@example.com","alice","example.com"],["bob@test.org","bob","test.org"]]}

re_find_object

Finds the first regex match and returns an object with named capture groups as keys (or numeric indices for unnamed groups). The key "0" contains the full match. Use for parsing structured text into fields.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_find_object("a(?P<foo>x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":{"0":"axxb","foo":"xx"}}
root.matches = this.value.re_find_object("(?P<key>\\w+):\\s+(?P<value>\\w+)")

# In:  {"value":"option1: value1"}
# Out: {"matches":{"0":"option1: value1","key":"option1","value":"value1"}}

re_match

Tests if a regular expression matches anywhere in a string, returning true or false. Use for validation or conditional routing based on patterns.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_match("[0-9]")

# In:  {"value":"there are 10 puppies"}
# Out: {"matches":true}

# In:  {"value":"there are ten puppies"}
# Out: {"matches":false}

re_replace

Replaces all regex matches with a replacement string that can reference capture groups using $1, $2, etc. Use for pattern-based transformations or data reformatting.

Parameters

Name Type Description

pattern

string

The pattern to match against.

value

string

The value to replace with.

re_replace_all

Replaces all regex matches with a replacement string that can reference capture groups using $1, $2, etc. Use for pattern-based transformations or data reformatting.

Parameters

Name Type Description

pattern

string

The pattern to match against.

value

string

The value to replace with.

Examples

root.new_value = this.value.re_replace_all("ADD ([0-9]+)","+($1)")

# In:  {"value":"foo ADD 70"}
# Out: {"new_value":"foo +(70)"}
root.masked = this.email.re_replace_all("(\\w{2})\\w+@", "$1***@")

# In:  {"email":"alice@example.com"}
# Out: {"masked":"al***@example.com"}

SQL

vector

Converts an array of numbers into a vector type suitable for insertion into SQL databases with vector/embedding support. This is commonly used with PostgreSQL’s pgvector extension for storing and querying machine learning embeddings, enabling similarity search and vector operations in your database.

Examples

Convert embeddings array to vector for pgvector storage:

root.embedding = this.embeddings.vector()
root.text = this.text

Process ML model output into database-ready vector format:

root.doc_id = this.id
root.vector_embedding = this.model_output.map_each(num -> num.number()).vector()

String manipulation

capitalize

Converts the first letter of each word in a string to uppercase (title case). Useful for formatting names, titles, and headings.

Examples

root.title = this.title.capitalize()

# In:  {"title":"the foo bar"}
# Out: {"title":"The Foo Bar"}
root.name = this.name.capitalize()

# In:  {"name":"alice smith"}
# Out: {"name":"Alice Smith"}

compare_argon2

Checks whether a string matches a hashed secret using Argon2.

Parameters

Name Type Description

hashed_secret

string

The hashed secret to compare with the input. This must be a fully-qualified string which encodes the Argon2 options used to generate the hash.

Examples

root.match = this.secret.compare_argon2("$argon2id$v=19$m=4096,t=3,p=1$c2FsdHktbWNzYWx0ZmFjZQ$RMUMwgtS32/mbszd+ke4o4Ej1jFpYiUqY6MHWa69X7Y")

# In:  {"secret":"there-are-many-blobs-in-the-sea"}
# Out: {"match":true}
root.match = this.secret.compare_argon2("$argon2id$v=19$m=4096,t=3,p=1$c2FsdHktbWNzYWx0ZmFjZQ$RMUMwgtS32/mbszd+ke4o4Ej1jFpYiUqY6MHWa69X7Y")

# In:  {"secret":"will-i-ever-find-love"}
# Out: {"match":false}

compare_bcrypt

Checks whether a string matches a hashed secret using bcrypt.

Parameters

Name Type Description

hashed_secret

string

The hashed secret value to compare with the input.

Examples

root.match = this.secret.compare_bcrypt("$2y$10$Dtnt5NNzVtMCOZONT705tOcS8It6krJX8bEjnDJnwxiFKsz1C.3Ay")

# In:  {"secret":"there-are-many-blobs-in-the-sea"}
# Out: {"match":true}
root.match = this.secret.compare_bcrypt("$2y$10$Dtnt5NNzVtMCOZONT705tOcS8It6krJX8bEjnDJnwxiFKsz1C.3Ay")

# In:  {"secret":"will-i-ever-find-love"}
# Out: {"match":false}

contains

Checks whether a string contains a substring and returns a boolean result.

Parameters

Name Type Description

value

unknown

A value to test against elements of the target.

Examples

root.has_foo = this.thing.contains("foo")

# In:  {"thing":"this foo that"}
# Out: {"has_foo":true}

# In:  {"thing":"this bar that"}
# Out: {"has_foo":false}

escape_html

Escapes special HTML characters (<, >, &, ', ") to make a string safe for HTML output. Use when embedding untrusted text in HTML to prevent XSS vulnerabilities.

Examples

root.escaped = this.value.escape_html()

# In:  {"value":"foo & bar"}
# Out: {"escaped":"foo &amp; bar"}
root.safe_html = this.user_input.escape_html()

# In:  {"user_input":"<script>alert('xss')</script>"}
# Out: {"safe_html":"&lt;script&gt;alert(&#39;xss&#39;)&lt;/script&gt;"}

escape_url_path

Encodes a string for safe use in URL path segments using percent-encoding. Unlike escape_url_query, spaces are encoded as %20 instead of +. Use when building URL paths with dynamic segments.

Examples

root.escaped = this.value.escape_url_path()

# In:  {"value":"foo & bar"}
# Out: {"escaped":"foo%20&%20bar"}
root.url = "https://example.com/docs/" + this.path.escape_url_path()

# In:  {"path":"my document.pdf"}
# Out: {"url":"https://example.com/docs/my%20document.pdf"}

escape_url_query

Encodes a string for safe use in URL query parameters. Converts spaces to + and special characters to percent-encoded values. Use when building URLs with dynamic query parameters.

Examples

root.escaped = this.value.escape_url_query()

# In:  {"value":"foo & bar"}
# Out: {"escaped":"foo+%26+bar"}
root.url = "https://example.com?search=" + this.query.escape_url_query()

# In:  {"query":"hello world!"}
# Out: {"url":"https://example.com?search=hello+world%21"}

filepath_join

Combines an array of path components into a single OS-specific file path using the correct separator (/ on Unix, \ on Windows). Use for constructing file paths from components.

Examples

root.path = this.path_elements.filepath_join()

# In:  {"path_elements":["/foo/","bar.txt"]}
# Out: {"path":"/foo/bar.txt"}

filepath_split

Separates a file path into directory and filename components, returning a two-element array [directory, filename]. Use for extracting the filename or directory from a full path.

Examples

root.path_sep = this.path.filepath_split()

# In:  {"path":"/foo/bar.txt"}
# Out: {"path_sep":["/foo/","bar.txt"]}

# In:  {"path":"baz.txt"}
# Out: {"path_sep":["","baz.txt"]}

format

Formats a string using Go’s printf-style formatting with the string as the format template. Supports all Go format verbs (%s, %d, %v, etc.). Use for building formatted strings from dynamic values.

Parameters

This method accepts any number of arguments.

Examples

root.foo = "%s(%v): %v".format(this.name, this.age, this.fingers)

# In:  {"name":"lance","age":37,"fingers":13}
# Out: {"foo":"lance(37): 13"}
root.message = "User %s has %v points".format(this.username, this.score)

# In:  {"username":"alice","score":100}
# Out: {"message":"User alice has 100 points"}

has_prefix

Tests if a string starts with a specified prefix. Returns true if the string begins with the prefix, false otherwise. Use for conditional logic based on string patterns.

Parameters

Name Type Description

value

string

The string to test.

Examples

root.t1 = this.v1.has_prefix("foo")
root.t2 = this.v2.has_prefix("foo")

# In:  {"v1":"foobar","v2":"barfoo"}
# Out: {"t1":true,"t2":false}

has_suffix

Tests if a string ends with a specified suffix. Returns true if the string ends with the suffix, false otherwise. Use for filtering or routing based on file extensions or string patterns.

Parameters

Name Type Description

value

string

The string to test.

Examples

root.t1 = this.v1.has_suffix("foo")
root.t2 = this.v2.has_suffix("foo")

# In:  {"v1":"foobar","v2":"barfoo"}
# Out: {"t1":false,"t2":true}

index_of

Finds the position of a substring within a string. Returns the zero-based index of the first occurrence, or -1 if not found. Useful for searching and string manipulation.

Parameters

Name Type Description

value

string

A string to search for.

Examples

root.index = this.thing.index_of("bar")

# In:  {"thing":"foobar"}
# Out: {"index":3}
root.index = content().index_of("meow")

# In:  the cat meowed, the dog woofed
# Out: {"index":8}

length

Returns the character count of a string.

Examples

root.foo_len = this.foo.length()

# In:  {"foo":"hello world"}
# Out: {"foo_len":11}

lowercase

Converts all letters in a string to lowercase. Use for case-insensitive comparisons, normalization, or formatting output.

Examples

root.foo = this.foo.lowercase()

# In:  {"foo":"HELLO WORLD"}
# Out: {"foo":"hello world"}
root.email = this.user_email.lowercase()

# In:  {"user_email":"User@Example.COM"}
# Out: {"email":"user@example.com"}

quote

Wraps a string in double quotes and escapes special characters (newlines, tabs, etc.) using Go escape sequences. Use for generating string literals or preparing strings for JSON-like formats.

Examples

root.quoted = this.thing.quote()

# In:  {"thing":"foo\nbar"}
# Out: {"quoted":"\"foo\\nbar\""}
root.literal = this.text.quote()

# In:  {"text":"hello\tworld"}
# Out: {"literal":"\"hello\\tworld\""}

repeat

Creates a new string by repeating the input string a specified number of times. Use for generating padding, separators, or test data.

Parameters

Name Type Description

count

integer

The number of times to repeat the string.

Examples

root.repeated = this.name.repeat(3)
root.not_repeated = this.name.repeat(0)

# In:  {"name":"bob"}
# Out: {"not_repeated":"","repeated":"bobbobbob"}
root.separator = "-".repeat(10)

# In:  {}
# Out: {"separator":"----------"}

replace

Replaces all occurrences of a substring with another string. Use for text transformation, cleaning data, or normalizing strings.

Parameters

Name Type Description

old

string

A string to match against.

new

string

A string to replace with.

replace_all

Replaces all occurrences of a substring with another string. Use for text transformation, cleaning data, or normalizing strings.

Parameters

Name Type Description

old

string

A string to match against.

new

string

A string to replace with.

Examples

root.new_value = this.value.replace_all("foo","dog")

# In:  {"value":"The foo ate my homework"}
# Out: {"new_value":"The dog ate my homework"}
root.clean = this.text.replace_all("  ", " ")

# In:  {"text":"hello  world  foo"}
# Out: {"clean":"hello world foo"}

replace_all_many

Performs multiple find-and-replace operations in sequence using an array of [old, new] pairs. More efficient than chaining multiple replace_all calls. Use for bulk text transformations.

Parameters

Name Type Description

values

array

An array of values, each even value will be replaced with the following odd value.

Examples

root.new_value = this.value.replace_all_many([
  "<b>", "&lt;b&gt;",
  "</b>", "&lt;/b&gt;",
  "<i>", "&lt;i&gt;",
  "</i>", "&lt;/i&gt;",
])

# In:  {"value":"<i>Hello</i> <b>World</b>"}
# Out: {"new_value":"&lt;i&gt;Hello&lt;/i&gt; &lt;b&gt;World&lt;/b&gt;"}

replace_many

Performs multiple find-and-replace operations in sequence using an array of [old, new] pairs. More efficient than chaining multiple replace_all calls. Use for bulk text transformations.

Parameters

Name Type Description

values

array

An array of values, each even value will be replaced with the following odd value.

reverse

Reverses the order of characters in a string. Unicode-aware for proper handling of multi-byte characters. Use for creating palindrome checks or reversing text data.

Examples

root.reversed = this.thing.reverse()

# In:  {"thing":"backwards"}
# Out: {"reversed":"sdrawkcab"}
root = content().reverse()

# In:  {"thing":"backwards"}
# Out: }"sdrawkcab":"gniht"{

slice

Extract a slice from a string by specifying two indices, a low and high bound, which selects a half-open range that includes the first character, but excludes the last one. If the second index is omitted then it defaults to the length of the input sequence.

Parameters

Name Type Description

low

integer

The low bound, which is the first element of the selection, or if negative selects from the end.

high (optional)

integer

An optional high bound.

Examples

root.beginning = this.value.slice(0, 2)
root.end = this.value.slice(4)

# In:  {"value":"foo bar"}
# Out: {"beginning":"fo","end":"bar"}

A negative low index can be used, indicating an offset from the end of the sequence. If the low index is greater than the length of the sequence then an empty result is returned:

root.last_chunk = this.value.slice(-4)
root.the_rest = this.value.slice(0, -4)

# In:  {"value":"foo bar"}
# Out: {"last_chunk":" bar","the_rest":"foo"}

slug

Converts a string into a URL-friendly slug by replacing spaces with hyphens, removing special characters, and converting to lowercase. Supports multiple languages for proper transliteration of non-ASCII characters.

Parameters

Name Type Description

lang (optional)

string

Default: "en".

Examples

Create a URL-friendly slug from a string with special characters:

root.slug = this.title.slug()

# In:  {"title":"Hello World! Welcome to Redpanda Connect"}
# Out: {"slug":"hello-world-welcome-to-redpanda-connect"}

Create a slug preserving French language rules:

root.slug = this.title.slug("fr")

# In:  {"title":"Café & Restaurant"}
# Out: {"slug":"cafe-et-restaurant"}

split

Splits a string into an array of substrings using a delimiter. Use for parsing CSV-like data, splitting paths, or breaking text into tokens.

Parameters

Name Type Description

delimiter

string

The delimiter to split with.

empty_as_null (optional)

bool

To treat empty substrings as null values. Default: false.

Examples

root.new_value = this.value.split(",")

# In:  {"value":"foo,bar,baz"}
# Out: {"new_value":["foo","bar","baz"]}
root.new_value = this.value.split(",", true)

# In:  {"value":"foo,,qux"}
# Out: {"new_value":["foo",null,"qux"]}
root.words = this.sentence.split(" ")

# In:  {"sentence":"hello world from bloblang"}
# Out: {"words":["hello","world","from","bloblang"]}

strip_html

Removes HTML tags from a string, returning only the text content. Useful for extracting plain text from HTML documents, sanitizing user input, or preparing content for text analysis. Optionally preserves specific HTML elements while stripping all others.

Parameters

Name Type Description

preserve (optional)

unknown

Optional array of HTML element names to preserve (e.g., ["strong", "em", "a"]). All other HTML tags will be removed.

Examples

Extract plain text from HTML content:

root.plain_text = this.html_content.strip_html()

# In:  {"html_content":"<p>Welcome to <strong>Redpanda Connect</strong>!</p>"}
# Out: {"plain_text":"Welcome to Redpanda Connect!"}

Preserve specific HTML elements while removing others:

root.sanitized = this.html.strip_html(["strong", "em"])

# In:  {"html":"<div><p>Some <strong>bold</strong> and <em>italic</em> text with a <script>alert('xss')</script></p></div>"}
# Out: {"sanitized":"Some <strong>bold</strong> and <em>italic</em> text with a "}

trim

Removes leading and trailing characters from a string. Without arguments, removes whitespace. With a cutset argument, removes any characters in the cutset. Use for cleaning user input or normalizing strings.

Parameters

Name Type Description

cutset (optional)

string

An optional string of characters to trim from the target value.

Examples

root.title = this.title.trim("!?")
root.description = this.description.trim()

# In:  {"description":"  something happened and its amazing! ","title":"!!!watch out!?"}
# Out: {"description":"something happened and its amazing!","title":"watch out"}

trim_prefix

Removes a specified prefix from the beginning of a string if present. If the string doesn’t start with the prefix, returns the string unchanged. Use for stripping known prefixes from identifiers or paths.

Parameters

Name Type Description

prefix

string

The leading prefix substring to trim from the string.

Examples

root.name = this.name.trim_prefix("foobar_")
root.description = this.description.trim_prefix("foobar_")

# In:  {"description":"unchanged","name":"foobar_blobton"}
# Out: {"description":"unchanged","name":"blobton"}

trim_suffix

Removes a specified suffix from the end of a string if present. If the string doesn’t end with the suffix, returns the string unchanged. Use for stripping file extensions or known suffixes.

Parameters

Name Type Description

suffix

string

The trailing suffix substring to trim from the string.

Examples

root.name = this.name.trim_suffix("_foobar")
root.description = this.description.trim_suffix("_foobar")

# In:  {"description":"unchanged","name":"blobton_foobar"}
# Out: {"description":"unchanged","name":"blobton"}

unescape_html

Converts HTML entities back to their original characters. Handles named entities (&, <), decimal (á), and hexadecimal (&xE1;) formats. Use for processing HTML content or decoding HTML-escaped data.

Examples

root.unescaped = this.value.unescape_html()

# In:  {"value":"foo &amp; bar"}
# Out: {"unescaped":"foo & bar"}
root.text = this.html.unescape_html()

# In:  {"html":"&lt;p&gt;Hello &amp; goodbye&lt;/p&gt;"}
# Out: {"text":"<p>Hello & goodbye</p>"}

unescape_url_path

Decodes URL path percent-encoding, converting %20 to spaces and other percent-encoded characters to their original values. Use for parsing URL path segments.

Examples

root.unescaped = this.value.unescape_url_path()

# In:  {"value":"foo%20&%20bar"}
# Out: {"unescaped":"foo & bar"}
root.filename = this.path.unescape_url_path()

# In:  {"path":"my%20document.pdf"}
# Out: {"filename":"my document.pdf"}

unescape_url_query

Decodes URL query parameter encoding, converting + to spaces and percent-encoded characters to their original values. Use for parsing URL query parameters.

Examples

root.unescaped = this.value.unescape_url_query()

# In:  {"value":"foo+%26+bar"}
# Out: {"unescaped":"foo & bar"}
root.search = this.param.unescape_url_query()

# In:  {"param":"hello+world%21"}
# Out: {"search":"hello world!"}

unicode_segments

Splits text into segments based on Unicode text segmentation rules. Returns an array of strings representing individual graphemes (visual characters), words (including punctuation and whitespace), or sentences. Handles complex Unicode correctly, including emoji with skin tone modifiers and zero-width joiners.

Parameters

Name Type Description

segmentation_type

string

Type of segmentation: "grapheme", "word", or "sentence"

Examples

Split text into sentences (preserves trailing spaces):

root.sentences = this.text.unicode_segments("sentence")

# In:  {"text":"Hello world. How are you?"}
# Out: {"sentences":["Hello world. ","How are you?"]}

Split text into grapheme clusters (handles complex emoji correctly):

root.graphemes = this.emoji.unicode_segments("grapheme")

# In:  {"emoji":"👨‍👩‍👧‍👦❤️"}
# Out: {"graphemes":["👨‍👩‍👧‍👦","❤️"]}

unquote

Removes surrounding quotes and interprets escape sequences (\n, \t, etc.) to their literal characters. Use for parsing quoted string literals.

Examples

root.unquoted = this.thing.unquote()

# In:  {"thing":"\"foo\\nbar\""}
# Out: {"unquoted":"foo\nbar"}
root.text = this.literal.unquote()

# In:  {"literal":"\"hello\\tworld\""}
# Out: {"text":"hello\tworld"}

uppercase

Converts all letters in a string to uppercase. Use for case-insensitive comparisons or formatting output.

Examples

root.foo = this.foo.uppercase()

# In:  {"foo":"hello world"}
# Out: {"foo":"HELLO WORLD"}
root.code = this.product_code.uppercase()

# In:  {"product_code":"abc-123"}
# Out: {"code":"ABC-123"}

Timestamp manipulation

parse_duration

Parses a Go-style duration string into nanoseconds. A duration string is a signed sequence of decimal numbers with unit suffixes like "300ms", "-1.5h", or "2h45m". Valid units: "ns", "us" (or "µs"), "ms", "s", "m", "h".

Examples

Parse microseconds to nanoseconds:

root.delay_for_ns = this.delay_for.parse_duration()

# In:  {"delay_for":"50us"}
# Out: {"delay_for_ns":50000}

Parse hours to seconds:

root.delay_for_s = this.delay_for.parse_duration() / 1000000000

# In:  {"delay_for":"2h"}
# Out: {"delay_for_s":7200}

parse_duration_iso8601

This method is in beta. Its behavior might change in a future release.

Parses an ISO 8601 duration string into nanoseconds. Format: "P[n]Y[n]M[n]DT[n]H[n]M[n]S" or "P[n]W". Example: "P3Y6M4DT12H30M5S" means 3 years, 6 months, 4 days, 12 hours, 30 minutes, 5 seconds. Supports fractional seconds with full precision (not just one decimal place).

Examples

Parse complex ISO 8601 duration to nanoseconds:

root.delay_for_ns = this.delay_for.parse_duration_iso8601()

# In:  {"delay_for":"P3Y6M4DT12H30M5S"}
# Out: {"delay_for_ns":110839937000000000}

Parse hours to seconds:

root.delay_for_s = this.delay_for.parse_duration_iso8601() / 1000000000

# In:  {"delay_for":"PT2H"}
# Out: {"delay_for_s":7200}

ts_add_iso8601

This method is in beta. Its behavior might change in a future release.

Adds an ISO 8601 duration to a timestamp with calendar-aware precision for years, months, and days. Useful when you need to add durations that account for variable month lengths or leap years.

Parameters

Name Type Description

duration

string

Duration in ISO 8601 format (e.g., "P1Y2M3D" for 1 year, 2 months, 3 days)

Examples

Add one year to a timestamp:

root.next_year = this.created_at.ts_add_iso8601("P1Y")

# In:  {"created_at":"2020-08-14T05:54:23Z"}
# Out: {"next_year":"2021-08-14T05:54:23Z"}

Add a complex duration with multiple units:

root.future_date = this.created_at.ts_add_iso8601("P1Y2M3DT4H5M6S")

# In:  {"created_at":"2020-01-01T00:00:00Z"}
# Out: {"future_date":"2021-03-04T04:05:06Z"}

ts_format

This method is in beta. Its behavior might change in a future release.

Formats a timestamp as a string using Go’s reference time format. Defaults to RFC 3339 if no format specified. The format uses "Mon Jan 2 15:04:05 -0700 MST 2006" as a reference. Accepts unix timestamps (with decimal precision) or RFC 3339 strings. Use ts_strftime for strftime-style formats.

Parameters

Name Type Description

format (optional)

string

The output format using Go’s reference time. Default: "2006-01-02T15:04:05.999999999Z07:00".

tz (optional)

string

Optional timezone (e.g., 'UTC', 'America/New_York'). Defaults to input timezone or local time for unix timestamps.

Examples

Format timestamp with custom format:

root.something_at = this.created_at.ts_format("2006-Jan-02 15:04:05")

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26"}

Format unix timestamp with timezone specification:

root.something_at = this.created_at.ts_format(format: "2006-Jan-02 15:04:05", tz: "UTC")

# In:  {"created_at":1597405526}
# Out: {"something_at":"2020-Aug-14 11:45:26"}

ts_parse

This method is in beta. Its behavior might change in a future release.

Parses a timestamp string using Go’s reference time format and outputs a timestamp object. The format uses "Mon Jan 2 15:04:05 -0700 MST 2006" as a reference - show how this reference time would appear in your format. Use ts_strptime for strftime-style formats instead.

Parameters

Name Type Description

format

string

The format of the input string using Go’s reference time.

Examples

Parse a date with abbreviated month name:

root.doc.timestamp = this.doc.timestamp.ts_parse("2006-Jan-02")

# In:  {"doc":{"timestamp":"2020-Aug-14"}}
# Out: {"doc":{"timestamp":"2020-08-14T00:00:00Z"}}

Parse a custom datetime format:

root.parsed = this.timestamp.ts_parse("Jan 2, 2006 at 3:04pm (MST)")

# In:  {"timestamp":"Aug 14, 2020 at 5:54am (UTC)"}
# Out: {"parsed":"2020-08-14T05:54:00Z"}

ts_round

This method is in beta. Its behavior might change in a future release.

Rounds a timestamp to the nearest multiple of the specified duration. Halfway values round up. Accepts unix timestamps (seconds with optional decimal precision) or RFC 3339 formatted strings.

Parameters

Name Type Description

duration

integer

A duration measured in nanoseconds to round by.

Examples

Round timestamp to the nearest hour:

root.created_at_hour = this.created_at.ts_round("1h".parse_duration())

# In:  {"created_at":"2020-08-14T05:54:23Z"}
# Out: {"created_at_hour":"2020-08-14T06:00:00Z"}

Round timestamp to the nearest minute:

root.created_at_minute = this.created_at.ts_round("1m".parse_duration())

# In:  {"created_at":"2020-08-14T05:54:23Z"}
# Out: {"created_at_minute":"2020-08-14T05:54:00Z"}

ts_strftime

This method is in beta. Its behavior might change in a future release.

Formats a timestamp as a string using strptime format specifiers (like %Y, %m, %d). Accepts unix timestamps (with decimal precision) or RFC 3339 strings. Supports %f for microseconds. Use ts_format for Go-style reference time formats.

Parameters

Name Type Description

format

string

The output format using strptime specifiers.

tz (optional)

string

Optional timezone. Defaults to input timezone or local time for unix timestamps.

Examples

Format timestamp with strftime specifiers:

root.something_at = this.created_at.ts_strftime("%Y-%b-%d %H:%M:%S")

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26"}

Format with microseconds using %f directive:

root.something_at = this.created_at.ts_strftime("%Y-%b-%d %H:%M:%S.%f", "UTC")

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26.371000"}

ts_strptime

This method is in beta. Its behavior might change in a future release.

Parses a timestamp string using strptime format specifiers (like %Y, %m, %d) and outputs a timestamp object. Use ts_parse for Go-style reference time formats instead.

Parameters

Name Type Description

format

string

The format string using strptime specifiers (e.g., %Y-%m-%d).

Examples

Parse date with abbreviated month using strptime format:

root.doc.timestamp = this.doc.timestamp.ts_strptime("%Y-%b-%d")

# In:  {"doc":{"timestamp":"2020-Aug-14"}}
# Out: {"doc":{"timestamp":"2020-08-14T00:00:00Z"}}

Parse datetime with microseconds using %f directive:

root.doc.timestamp = this.doc.timestamp.ts_strptime("%Y-%b-%d %H:%M:%S.%f")

# In:  {"doc":{"timestamp":"2020-Aug-14 11:50:26.371000"}}
# Out: {"doc":{"timestamp":"2020-08-14T11:50:26.371Z"}}

ts_sub

This method is in beta. Its behavior might change in a future release.

Calculates the duration in nanoseconds between two timestamps (t1 - t2). Returns a signed integer: positive if t1 is after t2, negative if t1 is before t2. Use .abs() for absolute duration.

Parameters

Name Type Description

t2

timestamp

The timestamp to subtract from the target timestamp.

Examples

Calculate absolute duration between two timestamps:

root.between = this.started_at.ts_sub("2020-08-14T05:54:23Z").abs()

# In:  {"started_at":"2020-08-13T05:54:23Z"}
# Out: {"between":86400000000000}

Calculate signed duration (can be negative):

root.duration_ns = this.end_time.ts_sub(this.start_time)

# In:  {"start_time":"2020-08-14T10:00:00Z","end_time":"2020-08-14T11:30:00Z"}
# Out: {"duration_ns":5400000000000}

ts_sub_iso8601

This method is in beta. Its behavior might change in a future release.

Subtracts an ISO 8601 duration from a timestamp with calendar-aware precision for years, months, and days. Useful when you need to subtract durations that account for variable month lengths or leap years.

Parameters

Name Type Description

duration

string

Duration in ISO 8601 format (e.g., "P1Y2M3D" for 1 year, 2 months, 3 days)

Examples

Subtract one year from a timestamp:

root.last_year = this.created_at.ts_sub_iso8601("P1Y")

# In:  {"created_at":"2020-08-14T05:54:23Z"}
# Out: {"last_year":"2019-08-14T05:54:23Z"}

Subtract a complex duration with multiple units:

root.past_date = this.created_at.ts_sub_iso8601("P1Y2M3DT4H5M6S")

# In:  {"created_at":"2021-03-04T04:05:06Z"}
# Out: {"past_date":"2020-01-01T00:00:00Z"}

ts_tz

This method is in beta. Its behavior might change in a future release.

Converts a timestamp to a different timezone while preserving the moment in time. Accepts unix timestamps (seconds with optional decimal precision) or RFC 3339 formatted strings.

Parameters

Name Type Description

tz

string

The timezone to change to. Use "UTC" for UTC, "Local" for local timezone, or an IANA Time Zone database location name like "America/New_York".

Examples

Convert timestamp to UTC timezone:

root.created_at_utc = this.created_at.ts_tz("UTC")

# In:  {"created_at":"2021-02-03T17:05:06+01:00"}
# Out: {"created_at_utc":"2021-02-03T16:05:06Z"}

Convert timestamp to a specific timezone:

root.created_at_ny = this.created_at.ts_tz("America/New_York")

# In:  {"created_at":"2021-02-03T16:05:06Z"}
# Out: {"created_at_ny":"2021-02-03T11:05:06-05:00"}

ts_unix

This method is in beta. Its behavior might change in a future release.

Converts a timestamp to a unix timestamp (seconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing seconds.

Examples

Convert RFC 3339 timestamp to unix seconds:

root.created_at_unix = this.created_at.ts_unix()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000}

Unix timestamp passthrough returns same value:

root.timestamp = this.ts.ts_unix()

# In:  {"ts":1257894000}
# Out: {"timestamp":1257894000}

ts_unix_micro

This method is in beta. Its behavior might change in a future release.

Converts a timestamp to a unix timestamp with microsecond precision (microseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing microseconds.

Examples

Convert timestamp to microseconds since epoch:

root.created_at_unix = this.created_at.ts_unix_micro()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000000000}

Preserve microsecond precision from timestamp:

root.precise_time = this.timestamp.ts_unix_micro()

# In:  {"timestamp":"2020-08-14T11:45:26.123456Z"}
# Out: {"precise_time":1597405526123456}

ts_unix_milli

This method is in beta. Its behavior might change in a future release.

Converts a timestamp to a unix timestamp with millisecond precision (milliseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing milliseconds.

Examples

Convert timestamp to milliseconds since epoch:

root.created_at_unix = this.created_at.ts_unix_milli()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000000}

Useful for JavaScript timestamp compatibility:

root.js_timestamp = this.event_time.ts_unix_milli()

# In:  {"event_time":"2020-08-14T11:45:26.123Z"}
# Out: {"js_timestamp":1597405526123}

ts_unix_nano

This method is in beta. Its behavior might change in a future release.

Converts a timestamp to a unix timestamp with nanosecond precision (nanoseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing nanoseconds.

Examples

Convert timestamp to nanoseconds since epoch:

root.created_at_unix = this.created_at.ts_unix_nano()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000000000000}

Preserve full nanosecond precision:

root.precise_time = this.timestamp.ts_unix_nano()

# In:  {"timestamp":"2020-08-14T11:45:26.123456789Z"}
# Out: {"precise_time":1597405526123456789}

Type coercion

array

Return an array containing the target value. If the value is already an array it is unchanged.

Examples

root.my_array = this.name.array()

# In:  {"name":"foobar bazson"}
# Out: {"my_array":["foobar bazson"]}

bool

Attempt to parse a value into a boolean. An optional argument can be provided, in which case if the value cannot be parsed the argument will be returned instead. If the value is a number then any non-zero value will resolve to true, if the value is a string then any of the following values are considered valid: 1, t, T, TRUE, true, True, 0, f, F, FALSE.

Parameters

Name Type Description

default (optional)

bool

An optional value to yield if the target cannot be parsed as a boolean.

Examples

root.foo = this.thing.bool()
root.bar = this.thing.bool(true)

bytes

Marshal a value into a byte array. If the value is already a byte array it is unchanged.

Examples

root.first_byte = this.name.bytes().index(0)

# In:  {"name":"foobar bazson"}
# Out: {"first_byte":102}

not_empty

Ensures that the given string, array or object value is not empty, and if so returns it, otherwise an error is returned.

Examples

root.a = this.a.not_empty()

# In:  {"a":"foo"}
# Out: {"a":"foo"}

# In:  {"a":""}
# Out: Error("failed assignment (line 1): field `this.a`: string value is empty")

# In:  {"a":["foo","bar"]}
# Out: {"a":["foo","bar"]}

# In:  {"a":[]}
# Out: Error("failed assignment (line 1): field `this.a`: array value is empty")

# In:  {"a":{"b":"foo","c":"bar"}}
# Out: {"a":{"b":"foo","c":"bar"}}

# In:  {"a":{}}
# Out: Error("failed assignment (line 1): field `this.a`: object value is empty")

not_null

Ensures that the given value is not null, and if so returns it, otherwise an error is returned.

Examples

root.a = this.a.not_null()

# In:  {"a":"foobar","b":"barbaz"}
# Out: {"a":"foobar"}

# In:  {"b":"barbaz"}
# Out: Error("failed assignment (line 1): field `this.a`: value is null")

number

Attempt to parse a value into a number. An optional argument can be provided, in which case if the value cannot be parsed into a number the argument will be returned instead.

Parameters

Name Type Description

default (optional)

float

An optional value to yield if the target cannot be parsed as a number.

Examples

root.foo = this.thing.number() + 10
root.bar = this.thing.number(5) * 10

string

Converts any value to its string representation. Numbers, booleans, and objects are converted to strings; existing strings are unchanged. Use for type coercion or creating string representations.

Examples

root.nested_json = this.string()

# In:  {"foo":"bar"}
# Out: {"nested_json":"{\"foo\":\"bar\"}"}
root.id = this.id.string()

# In:  {"id":228930314431312345}
# Out: {"id":"228930314431312345"}

timestamp

Attempt to parse a value into a timestamp. An optional argument can be provided, in which case if the value cannot be parsed into a timestamp the argument will be returned instead.

Parameters

Name Type Description

default (optional)

timestamp

An optional value to yield if the target cannot be parsed as a timestamp.

Examples

root.foo = this.ts.timestamp()
root.bar = this.none.timestamp(1234567890.timestamp())

type

Returns the type of a value as a string, providing one of the following values: string, bytes, number, bool, timestamp, array, object or null.

Examples

root.bar_type = this.bar.type()
root.foo_type = this.foo.type()

# In:  {"bar":10,"foo":"is a string"}
# Out: {"bar_type":"number","foo_type":"string"}
root.type = this.type()

# In:  "foobar"
# Out: {"type":"string"}

# In:  666
# Out: {"type":"number"}

# In:  false
# Out: {"type":"bool"}

# In:  ["foo", "bar"]
# Out: {"type":"array"}

# In:  {"foo": "bar"}
# Out: {"type":"object"}

# In:  null
# Out: {"type":"null"}
root.type = content().type()

# In:  foobar
# Out: {"type":"bytes"}
root.type = this.ts_parse("2006-01-02").type()

# In:  "2022-06-06"
# Out: {"type":"timestamp"}

Deprecated

format_timestamp

This method is deprecated and will be removed in a future version.

Formats a timestamp as a string using Go’s reference time format. Defaults to RFC 3339 if no format specified. The format uses "Mon Jan 2 15:04:05 -0700 MST 2006" as a reference. Accepts unix timestamps (with decimal precision) or RFC 3339 strings. Use ts_strftime for strftime-style formats.

Parameters

Name Type Description

format (optional)

string

The output format using Go’s reference time. Default: "2006-01-02T15:04:05.999999999Z07:00".

tz (optional)

string

Optional timezone (e.g., 'UTC', 'America/New_York'). Defaults to input timezone or local time for unix timestamps.

format_timestamp_strftime

This method is deprecated and will be removed in a future version.

Formats a timestamp as a string using strptime format specifiers (like %Y, %m, %d). Accepts unix timestamps (with decimal precision) or RFC 3339 strings. Supports %f for microseconds. Use ts_format for Go-style reference time formats.

Parameters

Name Type Description

format

string

The output format using strptime specifiers.

tz (optional)

string

Optional timezone. Defaults to input timezone or local time for unix timestamps.

format_timestamp_unix

This method is deprecated and will be removed in a future version.

Converts a timestamp to a unix timestamp (seconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing seconds.

format_timestamp_unix_micro

This method is deprecated and will be removed in a future version.

Converts a timestamp to a unix timestamp with microsecond precision (microseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing microseconds.

format_timestamp_unix_milli

This method is deprecated and will be removed in a future version.

Converts a timestamp to a unix timestamp with millisecond precision (milliseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing milliseconds.

format_timestamp_unix_nano

This method is deprecated and will be removed in a future version.

Converts a timestamp to a unix timestamp with nanosecond precision (nanoseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing nanoseconds.

parse_timestamp

This method is deprecated and will be removed in a future version.

Parses a timestamp string using Go’s reference time format and outputs a timestamp object. The format uses "Mon Jan 2 15:04:05 -0700 MST 2006" as a reference - show how this reference time would appear in your format. Use ts_strptime for strftime-style formats instead.

Parameters

Name Type Description

format

string

The format of the input string using Go’s reference time.

parse_timestamp_strptime

This method is deprecated and will be removed in a future version.

Parses a timestamp string using strptime format specifiers (like %Y, %m, %d) and outputs a timestamp object. Use ts_parse for Go-style reference time formats instead.

Parameters

Name Type Description

format

string

The format string using strptime specifiers (e.g., %Y-%m-%d).