gcp_vertex_ai_chat
Generates responses to messages in a chat conversation, using the Vertex AI API.
Introduced in version 4.34.0.
This processor sends prompts to your chosen large language model (LLM) and generates text from the responses, using the Vertex AI API.
For more information, see the Vertex AI documentation.
-
Common
-
Advanced
processor:
label: ""
gcp_vertex_ai_chat:
project: "" # No default (required)
credentials_json: "" # No default (optional)
location: "" # No default (required)
model: "" # No default (required)
prompt: "" # No default (optional)
history: "" # No default (optional)
attachment: "" # No default (optional)
temperature: 0 # No default (optional)
max_tokens: 0 # No default (optional)
response_format: text
tools: []processor:
label: ""
gcp_vertex_ai_chat:
project: "" # No default (required)
credentials_json: "" # No default (optional)
location: "" # No default (required)
model: "" # No default (required)
prompt: "" # No default (optional)
system_prompt: "" # No default (optional)
history: "" # No default (optional)
attachment: "" # No default (optional)
temperature: 0 # No default (optional)
max_tokens: 0 # No default (optional)
response_format: text
top_p: 0 # No default (optional)
top_k: 0 # No default (optional)
stop: [] # No default (optional)
presence_penalty: 0 # No default (optional)
frequency_penalty: 0 # No default (optional)
max_tool_calls: 10
tools: []Fields
attachment
Additional data like an image to send with the prompt to the model. The result of the mapping must be a byte array, and the content type is automatically detected.
Requires version 4.38.0 or later.
Type: string
# Examples:
attachment: 'root = this.image.decode("base64") # decode base64 encoded image'credentials_json
An optional field to set a Google Service Account Credentials JSON.
|
This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Secrets. |
Type: string
frequency_penalty
Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.
Type: float
history
Historical messages to include in the chat request. The result of the bloblang query should be an array of objects of the form of [{"role": "", "content":""}], where role is "user" or "model".
Requires version 4.56.0 or later.
Type: string
location
Specify the location of a fine tuned model. For base models, you can omit this field.
Type: string
# Examples:
location: us-central1max_tool_calls
The maximum number of sequential tool calls.
Requires version 4.56.0 or later.
Type: int
Default: 10
model
The name of the LLM to use. For a full list of models, see the Vertex AI Model Garden.
Type: string
# Examples:
model: gemini-1.5-pro-001
# ---
model: gemini-1.5-flash-001presence_penalty
Positive values penalize new tokens if they appear in the text already, increasing the model’s likelihood to include new topics.
Type: float
prompt
The user prompt you want to generate a response for. By default, the processor submits the entire payload of each message as a string.
This field supports interpolation functions.
Type: string
response_format
The format of the generated response. You must also prompt the model to output the appropriate response type.
Type: string
Default: text
Options: text, json
stop[]
Sets the stop sequences to use. When the model encounters one of these sequences, it stops generating text and returns the final response.
Type: array<string>
system_prompt
The system prompt to submit along with the user prompt.
This field supports interpolation functions.
Type: string
tools[]
The tools to allow the LLM to invoke. This allows building subpipelines that the LLM can choose to invoke to execute agentic-like actions.
Requires version 4.56.0 or later.
Type: array<object>
Default: []
tools[].description
A description of this tool. The LLM uses it to decide whether to invoke the tool.
Type: string
tools[].parameters.properties
The parameters the LLM can provide when it invokes this tool, keyed by parameter name.
Type: object
tools[].parameters.properties.enum[]
The values this parameter is limited to. Leave empty to accept any value.
Type: array<string>
Default: []
tools[].parameters.required[]
The names of the parameters the LLM must provide when it invokes this tool.
Type: array<string>
Default: []
Examples
Use processors as tool calls
This example allows gemini to execute a subpipeline as a tool call to get more data.
input:
generate:
count: 1
mapping: |
root = "What is the weather like in Chicago?"
pipeline:
processors:
- gcp_vertex_ai_chat:
model: gemini-2.5-flash-preview-05-20
project: my-project
location: us-central1
prompt: "${!content().string()}"
tools:
- name: GetWeather
description: "Retrieve the weather for a specific city"
parameters:
required: ["city"]
properties:
city:
type: string
description: the city to lookup the weather for
processors:
- http:
verb: GET
url: 'https://wttr.in/${!this.city}?T'
headers:
# Spoof curl user-agent to get a plaintext text
User-Agent: curl/8.11.1
output:
stdout: {}