Cloud
Also in Self-Managed

gcp_vertex_ai_chat

Generates responses to messages in a chat conversation, using the Vertex AI API.

This processor sends prompts to your chosen large language model (LLM) and generates text from the responses, using the Vertex AI API.

For more information, see the Vertex AI documentation.

  • Common

  • Advanced

processor:
  label: ""
  gcp_vertex_ai_chat:
    project: "" # No default (required)
    credentials_json: "" # No default (optional)
    location: "" # No default (required)
    model: "" # No default (required)
    prompt: "" # No default (optional)
    history: "" # No default (optional)
    attachment: "" # No default (optional)
    temperature: 0 # No default (optional)
    max_tokens: 0 # No default (optional)
    response_format: text
    tools: []
processor:
  label: ""
  gcp_vertex_ai_chat:
    project: "" # No default (required)
    credentials_json: "" # No default (optional)
    location: "" # No default (required)
    model: "" # No default (required)
    prompt: "" # No default (optional)
    system_prompt: "" # No default (optional)
    history: "" # No default (optional)
    attachment: "" # No default (optional)
    temperature: 0 # No default (optional)
    max_tokens: 0 # No default (optional)
    response_format: text
    top_p: 0 # No default (optional)
    top_k: 0 # No default (optional)
    stop: [] # No default (optional)
    presence_penalty: 0 # No default (optional)
    frequency_penalty: 0 # No default (optional)
    max_tool_calls: 10
    tools: []

Fields

attachment

Additional data like an image to send with the prompt to the model. The result of the mapping must be a byte array, and the content type is automatically detected.

Type: string

# Examples:
attachment: 'root = this.image.decode("base64") # decode base64 encoded image'

credentials_json

An optional field to set a Google Service Account Credentials JSON.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

frequency_penalty

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.

Type: float

history

Historical messages to include in the chat request. The result of the bloblang query should be an array of objects of the form of [{"role": "", "content":""}], where role is "user" or "model".

Type: string

location

Specify the location of a fine tuned model. For base models, you can omit this field.

Type: string

# Examples:
location: us-central1

max_tokens

The maximum number of output tokens to generate per message.

Type: int

max_tool_calls

The maximum number of sequential tool calls.

Type: int

Default: 10

model

The name of the LLM to use. For a full list of models, see the Vertex AI Model Garden.

Type: string

# Examples:
model: gemini-1.5-pro-001

# ---

model: gemini-1.5-flash-001

presence_penalty

Positive values penalize new tokens if they appear in the text already, increasing the model’s likelihood to include new topics.

Type: float

project

The GCP project ID to use.

Type: string

prompt

The user prompt you want to generate a response for. By default, the processor submits the entire payload of each message as a string.

This field supports interpolation functions.

Type: string

response_format

The format of the generated response. You must also prompt the model to output the appropriate response type.

Type: string

Default: text

Options: text, json

stop[]

Sets the stop sequences to use. When the model encounters one of these sequences, it stops generating text and returns the final response.

Type: array<string>

system_prompt

The system prompt to submit along with the user prompt.

This field supports interpolation functions.

Type: string

temperature

Controls the randomness of predictions.

Type: float

tools[]

The tools to allow the LLM to invoke. This allows building subpipelines that the LLM can choose to invoke to execute agentic-like actions.

Type: array<object>

Default: []

tools[].description

A description of this tool. The LLM uses it to decide whether to invoke the tool.

Type: string

tools[].name

The name of this tool.

Type: string

tools[].parameters

The parameters the LLM needs to provide to invoke this tool.

Type: object

tools[].parameters.properties

The parameters the LLM can provide when it invokes this tool, keyed by parameter name.

Type: object

tools[].parameters.properties.description

A description of this parameter.

Type: string

tools[].parameters.properties.enum[]

The values this parameter is limited to. Leave empty to accept any value.

Type: array<string>

Default: []

tools[].parameters.properties.type

The type of this parameter.

Type: string

tools[].parameters.required[]

The names of the parameters the LLM must provide when it invokes this tool.

Type: array<string>

Default: []

tools[].processors[]

The processors to run when the LLM invokes this tool. They receive a message whose payload is the tool call arguments as a JSON object, and their output is returned to the LLM as the tool result.

Type: array<processor>

top_k

Enables top-k sampling (optional).

Type: float

top_p

Enables nucleus sampling (optional).

Type: float

Examples

Use processors as tool calls

This example allows gemini to execute a subpipeline as a tool call to get more data.

input:
  generate:
    count: 1
    mapping: |
      root = "What is the weather like in Chicago?"
pipeline:
  processors:
    - gcp_vertex_ai_chat:
        model: gemini-2.5-flash-preview-05-20
        project: my-project
        location: us-central1
        prompt: "${!content().string()}"
        tools:
          - name: GetWeather
            description: "Retrieve the weather for a specific city"
            parameters:
              required: ["city"]
              properties:
                city:
                  type: string
                  description: the city to lookup the weather for
            processors:
              - http:
                  verb: GET
                  url: 'https://wttr.in/${!this.city}?T'
                  headers:
                    # Spoof curl user-agent to get a plaintext text
                    User-Agent: curl/8.11.1
output:
  stdout: {}