Scale a BYOC Cluster
Scale a BYOC cluster up or down as your workload changes, without contacting Redpanda Support. You can add or remove brokers to absorb short-term changes in traffic, or move the cluster to a different machine type to right-size it for a sustained change in throughput. Self-service scaling is available for BYOC clusters in organizations on usage-based billing.
After reading this page, you will be able to:
-
Choose between horizontal and vertical scaling for a workload change
-
Scale a cluster up or down from the Redpanda Cloud UI or the Control Plane API
-
Explain what happens to your cluster and clients during a scaling operation
Choose a scaling method
You can scale a cluster in two ways. The Cloud UI tells you which one applies before you save.
| Method | When to use it | What changes | Typical duration |
|---|---|---|---|
Horizontal scaling (scale out or in) |
Changes in traffic within a day, such as business-hours peaks or nightly batch jobs. |
Adds or removes brokers of the current machine type. |
About 20 minutes to scale out. Scaling in takes longer, typically about an hour, because brokers must decommission. |
Vertical scaling (scale up or down) |
Changes in throughput that last days or longer, such as steady growth, onboarding a new workload, or a weekday versus weekend pattern. |
Replaces the brokers with a larger or smaller machine type. Redpanda provisions brokers of the new machine type and replaces the existing brokers one at a time. |
About 10 minutes per broker, counting the brokers in the new configuration: around an hour for a small cluster, up to about five hours for the largest. |
These durations are estimates. Large or busy clusters, and clusters with a lot of local data not yet offloaded to Tiered Storage, take longer. The Cloud UI shows an estimate for your specific change before you save.
In the Cloud UI, describe your workload and Redpanda recommends a machine type and broker count. In the Control Plane API and Terraform, you select the machine type through the cluster’s usage tier (throughput_tier) and the broker count through redpanda_node_count. See Size and Price a BYOC Cluster for how workload sizing works.
Before you scale
-
The cluster must be in the Ready state. You cannot start a scaling operation while another operation is running, and you cannot make other changes to the cluster, such as enabling a feature or editing cluster configuration, until scaling completes.
-
A cluster has at least 3 brokers and at most 24. The upper bound depends on the machine type. Multi-AZ clusters scale in steps of 3 brokers so that each availability zone keeps the same number of brokers.
-
You can move to any machine type in the same cloud provider, machine family, and CPU architecture as your current one. In the API, for example, a cluster on
tier-2-aws-v3-armcan move directly totier-5-aws-v3-arm. To move to a different machine family or CPU architecture, contact Redpanda Support. -
Redpanda runs preflight checks before it accepts a scaling request. For a change that reduces capacity, the checks reject the request if the cluster has firing alerts, if the remaining brokers would not have enough storage or partition capacity for the current data, if a topic’s replication factor would exceed the new broker count, or if recent throughput or CPU usage is too high for the smaller configuration. The request fails immediately with a message that names the check, so you can fix the condition and try again.
-
On GCP, make sure your project has enough Local SSD quota for the larger broker count or machine type before you scale out or up. See Create a BYOC Cluster on GCP.
Scale a cluster in the Cloud UI
-
Log in to Redpanda Cloud and open your cluster.
-
Click Settings.
-
In the Cluster card, find Scale cluster and click Edit. The Performance configuration panel opens and shows your current machine type, broker count, and RPU.
The Edit button is available only while the cluster is in the Ready state.
-
Under Size your workload, describe the workload you want the cluster to handle:
-
On the Standard tab, enter the Number of Partitions, Ingress, and Egress. Optionally, expand Additional options to set a Latency target, Producer count, and Consumer count.
-
On the Advanced tab, enter an RPU Rate if you already know the capacity you need.
-
-
Under Recommended configuration, review the machine type and broker count that Redpanda recommends and its calculated RPU. To use a different configuration, select one of the alternatives.
-
Review Estimated costs and the Comparison table, which shows the current and target usage tier, machine type, RPU, node count, and maximum ingress, egress, partitions, and connections. A Replaced badge on the machine type means that every broker is replaced.
-
Read the change summary above Save changes. It states whether the change is horizontal or vertical scaling and gives an estimated duration. If only the tier’s limits change, with the same machine type and broker count, the summary labels the change Tier change instead.
-
Click Save changes. Scaling starts immediately.
While the operation runs, the cluster state shows Upgrading on the Clusters page and on the cluster overview, and returns to Ready when scaling completes. See Cluster State.
Scale a cluster with the Control Plane API
To scale a cluster programmatically, send a PATCH /v1/clusters/{id} request that sets redpanda_node_count, throughput_tier, or both, and name the fields you are changing in update_mask. Scaling starts as soon as the request is accepted.
To change the broker count within the current tier (horizontal scaling):
curl -X PATCH "https://api.redpanda.com/v1/clusters/<cluster-id>?update_mask=redpanda_node_count" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"redpanda_node_count": 6
}'To change the tier and machine type (vertical scaling):
curl -X PATCH "https://api.redpanda.com/v1/clusters/<cluster-id>?update_mask=throughput_tier" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"throughput_tier": "tier-3-aws-v3-arm"
}'To change both in one request, set both fields. Redpanda validates the broker count against the target tier:
curl -X PATCH "https://api.redpanda.com/v1/clusters/<cluster-id>?update_mask=throughput_tier,redpanda_node_count" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"throughput_tier": "tier-3-aws-v3-arm",
"redpanda_node_count": 9
}'The request returns HTTP 202 with a long-running operation of type TYPE_UPDATE_CLUSTER. Poll GET /v1/operations/{id} until it completes. While scaling runs, GET /v1/clusters/{id} reports STATE_UPGRADING and already returns the new tier and broker count. The state returns to STATE_READY when scaling completes.
Errors
Redpanda rejects a scaling request synchronously when it cannot be fulfilled, so the response tells you what to fix. Common responses:
| Status | Message | Cause and action |
|---|---|---|
403 ( |
|
Your organization is not on usage-based billing. Contact your Redpanda account team. |
400 ( |
|
Another operation is running. Wait for the cluster to return to Ready. |
400 ( |
|
The minimum is 3 brokers. |
400 ( |
|
The count exceeds the maximum for the current machine type. Choose a lower count or a larger machine type. |
400 ( |
|
The target machine type is in a different machine family or CPU architecture. Contact Redpanda Support to migrate. |
400 ( |
|
The target configuration is not offered in the cluster’s region. See BYOC tiers and regions. |
400 ( |
|
A preflight check failed. Resolve the condition named in the message and retry. |
400 ( |
|
An earlier scaling operation stopped because of a cloud provider stockout or quota limit. Contact Redpanda Support. |
Most error responses also include a machine-readable reason in the error details, such as REASON_SCALING_REDPANDA_NODE_COUNT_TOO_HIGH or REASON_SCALING_UNSUPPORTED_THROUGHPUT_TIER_MIGRATION_PATH.
What happens during scaling
Your cluster stays available throughout a scaling operation and its endpoints do not change. Expect the following:
-
Partition leadership moves as brokers are added, replaced, or removed, and partitions rebalance across the resulting set of brokers. Clients see leader elections and dropped connections. All mainstream Kafka client libraries reconnect and retry automatically.
-
When scaling out horizontally, the new brokers join first, and partitions rebalance onto them in the background. When scaling in, Redpanda decommissions the surplus brokers and moves their partitions to the remaining brokers before removing them.
-
When scaling vertically, Redpanda provisions brokers of the new machine type, then drains and decommissions the existing brokers one at a time, and removes the original brokers when all data has moved.
-
Metrics might show a gap of 5 to 10 minutes near the end of a scaling operation if the auxiliary nodes that run monitoring are also replaced.
-
The cluster shows Upgrading and rejects other configuration changes until scaling completes.
If the cloud provider cannot supply the requested machines because of a stockout or a quota limit, the operation pauses with the brokers provisioned so far, and the cluster stays in Upgrading. The long-running operation eventually times out, and further scaling requests are rejected with a message that scaling is paused. Contact Redpanda Support to resume or roll back the change.
Limits
-
Self-service scaling applies to BYOC clusters in organizations on usage-based billing. For Dedicated clusters, or if you want Redpanda to run the change for you, see Tier migrations.
-
The broker count range, the machine family and CPU architecture rule, and the one-operation-at-a-time rule are described in Before you scale.
-
Some older clusters return
cluster does not support product/node count change - contact support. Contact Redpanda Support to scale these clusters.
