Skip to main content
BYOK8s is currently in early access. To enable BYOK8s for your organization, contact our support team.
The Bring Your Own Kubernetes (BYOK8s) plan lets you run RisingWave in your own Kubernetes cluster while RisingWave manages the database workloads. Unlike BYOC where RisingWave manages the entire data plane infrastructure, BYOK8s gives you full control over the underlying cloud resources — networking, IAM, storage, and the Kubernetes cluster itself — while RisingWave handles the Kubernetes-level workloads and cluster operations.

BYOC vs BYOK8s

Architecture overview

In a BYOK8s environment, RisingWave deploys and manages the following components in your Kubernetes cluster:
  • CloudAgent: Handles operations sent by the RisingWave control plane (cluster provisioning, scaling, upgrades).
  • RWProxy: Routes PostgreSQL protocol traffic from the control plane and user clients to the appropriate RisingWave instances.
  • RisingWave Operator: Manages RisingWave Custom Resource lifecycle in Kubernetes.
  • Self-hosted telemetry stack: VictoriaMetrics (metrics), Loki (logs), and Grafana Alloy (collection) — deployed automatically inside the BYOK8s cluster. No external observability services are required.
Communication between the RisingWave control plane and your BYOK8s cluster is established via private network connectivity (AWS PrivateLink), ensuring all traffic stays off the public internet.

Prerequisites

Before setting up a BYOK8s environment, you must provision the following resources in your cloud account.
Reference Terraform examples are available at risingwavelabs/risingwave-byok-terraform-example. They provision all of the prerequisites below and generate the config files used by the rwc CLI. Fork and adapt to your network/security requirements.
  • aws/base_env — VPC, EKS, S3, KMS, IAM, NLBs + VPC Endpoint Services
  • aws/k8s_addons — cert-manager and mode-specific node pools; standard EKS also installs the AWS Load Balancer Controller and Karpenter (emits byok_config.yaml)
  • aws/tenant_resources — per-cluster RDS metastore + IAM role (emits byok_tenant_config.yaml)

1. Kubernetes cluster

  • Provider: Amazon EKS
  • Version: Kubernetes 1.32 or higher
  • Region: Must be in the same AWS region as the RisingWave Cloud control plane
  • Cluster mode: Standard EKS or EKS Auto Mode
Choose the cluster mode before you register the BYOK8s environment. The aws.eks_auto_mode setting is stored when you run rwc byok create and cannot be changed with rwc byok update. RisingWave does not support an in-place migration between standard EKS and EKS Auto Mode; create a new BYOK8s environment to change modes.
The following Kubernetes namespaces are reserved for RisingWave-managed components. Do not create or use these namespaces for your own workloads — RisingWave will create and manage them during BYOK8s environment setup and cluster provisioning.

2. Cluster dependencies

Install cert-manager v1.19.2 or later in both cluster modes. The remaining dependencies differ by mode:In EKS Auto Mode, RisingWave creates native eks.amazonaws.com/v1 TargetGroupBinding resources. In a standard EKS cluster, RisingWave creates elbv2.k8s.aws/v1beta1 TargetGroupBinding resources that the self-managed AWS Load Balancer Controller reconciles.

3. Object storage

Provision two separate storage buckets in the same region as your Kubernetes cluster.
We recommend separate buckets so that data and logs can have independent lifecycle policies, access controls, and retention. The data store typically contains sensitive cluster state, while the log store contains observability data with different access and retention requirements.
Provide the S3 bucket ARNs for both buckets.

4. Encryption at rest

Provide a KMS key ARN for EBS encryption. This is used by VictoriaMetrics and Loki persistent volumes, RisingWave compute cache storage, and encrypted EKS worker-node storage.
The KMS key policy must allow the key to be used for EBS encryption. Otherwise, worker nodes can fail to join the cluster or become NotReady, persistent volume claims for VictoriaMetrics, Loki, and the RisingWave compute cache can remain Pending, and rwc byok apply will fail.
For standard EKS, grant the IAM principal used by the Amazon EBS CSI controller the required KMS permissions.For EKS Auto Mode, the key policy must grant:
  • Allow the EKS cluster IAM role and the AWSServiceRoleForAutoScaling service-linked role to encrypt, decrypt, and re-encrypt data; generate data keys; describe the key; and create grants for AWS resources.
  • Allow the EKS cluster IAM role to create, list, and revoke grants for AWS resources.
  • Allow principals in the same AWS account that are authorized to use EBS to use the key through EC2. Restrict this statement with kms:CallerAccount and kms:ViaService set to ec2.<region>.amazonaws.com.
The IAM role used by an Auto Mode NodeClass does not need to be an explicit key-policy principal when the account-scoped EBS condition is present. See the reference Terraform example for a working policy.

5. Terraform state backend

Provision an S3 bucket and a DynamoDB table for storing BYOK8s environment Terraform state and state locking. CloudAgent uses this backend to manage Terraform resources inside your Kubernetes cluster, and the IAM policy in the next step references both.

6. Identity and access management

You must create the following IAM roles:A. Setup IAM (user context)This role is used by the person or CI/CD pipeline running the BYOK8s setup commands. It is not used by any in-cluster workloads.
  • Must have eks:DescribeCluster permission on the target EKS cluster.
  • Must have an access entry in the EKS cluster with the AmazonEKSClusterAdminPolicy access policy attached.
B. CloudAgent IAM (service context)This role is assumed by CloudAgent via IRSA. CloudAgent needs access to both the data store bucket (for cluster state operations) and the Terraform state backend (S3 bucket plus DynamoDB lock table) that it uses to manage in-cluster resources. Minimum permissions:
Where $TFSTATE_BUCKET and $TFLOCK_TABLE correspond to the S3 bucket and DynamoDB table provisioned in Terraform state backend.C. Loki IAM (service context)This role is assumed by Loki via IRSA for reading and writing logs to S3. Minimum permissions:

7. Network connectivity

Provision two internal Network Load Balancers with VPC Endpoint Services for PrivateLink connectivity from the RisingWave control plane.CloudAgent NLB — one target group per port:RWProxy NLB — one target group per port. The webhook listener port differs from its target port (the control plane reaches it on 443 over TLS; the RWProxy pod listens on 4580); pgwire and metrics use the same port on both sides.For each NLB, create a VPC Endpoint Service and allow the RisingWave control plane AWS account (600598779918) as an allowed principal.

EKS Auto Mode load-balancing requirements

EKS Auto Mode reconciles the pre-created target groups through its native TargetGroupBinding API. In addition to the shared NLB requirements above:
  • Add the tag eks:eks-cluster-name=<cluster-name> to all five target groups.
  • Attach a target-traffic security group to both NLBs, and allow outbound TCP traffic from it to ports 40001, 40090, 4566, 4580, and 9099.
  • Allow inbound TCP traffic from the target-traffic security group to the EKS cluster primary security group on the same five ports.
  • If clients connect directly to the RWProxy NLB, attach a separate client-access security group only to that NLB and allow inbound TCP traffic on port 4566 from the approved client CIDRs or security groups.
  • Ensure every Auto Mode NodeClass selects the EKS cluster primary security group.
  • Set the NLB attribute enforce_security_group_inbound_rules_on_private_link_traffic to off for the PrivateLink traffic used by this setup.
The standard EKS bindings carry their own target network rules, so these NLB security-group resources are Auto Mode-only. The AWS reference example configures the control-plane PrivateLink path for both modes. In Auto Mode, it attaches the target-traffic security group to both NLBs and a dedicated client-access security group only to the RWProxy NLB. The client-access security group allows direct RWProxy connections from the VPC CIDR by default; set rwproxy_additional_client_cidrs for clients that arrive through peering, Transit Gateway, or another routed network.

Create a BYOK8s environment

BYOK8s uses the rwc CLI to provision and manage environments. Before continuing, install and authenticate the CLI by following Install the RisingWave Cloud CLI.

Step 1: Prepare the configuration file

Create a YAML configuration file with your infrastructure details:
For a standard EKS cluster, leave aws.eks_auto_mode set to false. For EKS Auto Mode, set it to true. The value defaults to false when omitted, must match the existing cluster mode when you register the environment, and cannot be changed later with rwc byok update.
You can optionally configure pod scheduling for different workload types and add custom tags:
See Node pool resource requirements for sizing guidance on each workload category.

Step 2: Register the environment

Step 3: Deploy resources

This deploys all RisingWave-managed components to your EKS cluster via Terraform. The control plane then establishes PrivateLink connectivity and verifies the environment. Once complete, the environment status transitions to Ready.
You can run rwc byok prepare --name <env-name> to download the Terraform modules without applying them — useful for reviewing changes before deployment.

Create a RisingWave cluster

Creating a RisingWave cluster in a BYOK8s environment uses a two-phase provisioning flow. This is because the IRSA trust policy for the cluster’s IAM role requires the Kubernetes namespace and service account name, which are only allocated after the cluster is initially created.
Cluster creation in a BYOK8s environment is currently available only via the rwc CLI. RisingWave Cloud portal support is planned but not yet implemented. Once the cluster is provisioned, you can manage it (including rescaling components and adjusting replicas) from the RisingWave Cloud portal as usual.

Phase 1: Create the cluster

Create the cluster via the rwc CLI. A BYOK8s cluster must use --tier BYOK together with per-component sizing flags, and the --env flag must point to the BYOK8s environment registered with rwc byok create. The metastore is not configured at this step — the customer-managed PostgreSQL connection details are supplied in Phase 2 via rwc cluster byok-config.
Where:
  • --name — name of the new RisingWave cluster.
  • --tier BYOK — required tier for clusters running in a BYOK8s environment.
  • --env — name of the BYOK8s environment registered with rwc byok create.
  • --compute / --compactor / --frontend / --meta — component-type IDs that match instance shapes available in your Kubernetes cluster. Component IDs follow the pattern p-<X>c<Y>g, where X is the number of CPU cores and Y is the amount of memory in GiB (for example, p-4c16g means 4 CPU and 16 GiB memory). The values above are a small example sizing (1 replica each at p-4c16g for compute and p-2c8g for the rest); replace them to match your workload. Run rwc cluster create --help for the full flag list.
  • --*-replica — replica count for each component.
The cluster starts in AwaitingConfig status. The success message from rwc cluster create includes the cluster’s UUID and the exact rwc cluster byok-config command to run next; you can also list existing clusters and their UUIDs with rwc cluster list. At this point, the control plane has allocated the Kubernetes namespace and service account name. Retrieve them with:
Look for the Resource Namespace and Service Account fields in the output. You will need these values for the IAM trust policy in the next step.

Phase 2: Provision IAM and metastore, then configure

  1. Create an IAM role for the RisingWave cluster with an IRSA trust policy. Use the Resource Namespace and Service Account values from the rwc cluster describe output in Phase 1:
    Grant S3 access to the cluster’s data and backup prefixes in the data store bucket. RisingWave stores cluster state under data-$RESOURCE_NAMESPACE/ and meta backups under data-$RESOURCE_NAMESPACE-backup/, so both prefixes must be included:
  2. Provision a PostgreSQL database as the metadata store. We recommend using Amazon RDS. Ensure connectivity from the EKS cluster pods to the database.
    Choose the metastore password using only these characters: uppercase and lowercase letters (A-Z, a-z), digits (0-9), and the four symbols underscore (_), tilde (~), period (.), and hyphen (-). No other characters are allowed — in particular, spaces and symbols such as @, /, and : are not. RisingWave Cloud embeds this password in the metastore connection URL, so a password containing unsupported characters is rejected by rwc cluster byok-config with an HTTP 400 error, even though your database may accept it. Fixing that means rotating the password and reconfiguring the database, so pick a compliant one before you create the database.
  3. Submit the configuration via CLI:
    Config file format:
    You can set the RWC_BYOK_METASTORE_PASSWORD environment variable instead of including the password in the YAML file. The environment variable takes precedence.
This transitions the cluster to Creating status and triggers provisioning. The metastore password is encrypted at rest.

Connect to a RisingWave cluster

There are two ways to connect to a RisingWave cluster running in a BYOK8s environment. Use the in-portal SQL console — no network setup is required. See Console overview for details.

Option 2: Direct connection via RWProxy NLB

For programmatic access (e.g., from your own applications, BI tools, or psql), connect to the RWProxy NLB you provisioned in the prerequisites.
The RWProxy NLB is internal-only. Clients must be inside the same VPC as the BYOK8s environment, or reach the NLB via VPC peering / Transit Gateway.
For EKS Auto Mode, attach a client-access security group only to the RWProxy NLB before using direct connections. Allow inbound TCP traffic on port 4566 from the approved client CIDRs or security groups. The reference Terraform example allows its VPC CIDR by default and accepts additional routed CIDRs through rwproxy_additional_client_cidrs. Disabling NLB security-group enforcement for PrivateLink traffic does not allow direct VPC client traffic.
Because a single RWProxy NLB serves multiple clusters in the BYOK8s environment, you must include the cluster identifier (the Kubernetes namespace allocated to your cluster in Phase 1) in the connection. Retrieve it via:
The cluster identifier can be passed via the options field, the host (SNI), or the username. See Connection errors → Tenant identifier methods for all three methods. Example using the options field:
%3D is the URL-encoded form of =. If your client does not require URL encoding, use --tenant=<cluster-namespace> directly. Where:
  • <rwproxy-nlb-dns> — DNS name of the RWProxy NLB (the load balancer behind rwproxy_target_group_arn)
  • 4566 — RWProxy PostgreSQL port
  • <database> — database name (default: dev)
  • <cluster-namespace> — the Resource Namespace from rwc cluster describe
  • <username> / <password> — credentials created via rwc cluster dbuser create

Node pool resource requirements

If you use dedicated node pools for different workload types (via the customized_settings.scheduling configuration), the following minimum resources are required: The same workload categories and minimum sizing apply to both standard EKS and EKS Auto Mode:
DaemonSet scheduling (Grafana Alloy) is automatically derived from the union of all workload tolerations — no separate configuration is needed.

Manage a BYOK8s environment

List environments

View environment details

Update an environment

To update a BYOK8s environment (e.g., to apply a new version or change custom settings):
The cluster mode cannot be updated. To switch between standard EKS and EKS Auto Mode, create a new BYOK8s environment configured for the target mode.

Delete a BYOK8s environment

  1. Delete all RisingWave clusters running in the environment.
  2. Terminate the environment (control plane side):
  3. Delete the Kubernetes resources and control plane record:
  4. Clean up the cloud resources you provisioned (NLBs, VPC Endpoint Services, IAM roles, storage buckets, etc.).

Shared responsibility

BYOK8s is a shared responsibility model. The table below outlines what each party is responsible for.

RisingWave responsibilities

  • All cluster and project-level operations available on the RisingWave Cloud portal.
  • 24/7 monitoring backed by the RisingWave support and on-call teams.
  • Version and configuration management of all deployed workloads (CloudAgent, RWProxy, Operator, telemetry stack).

Customer responsibilities

  • Kubernetes cluster and worker nodes (including upgrades to the minimum supported version).
  • Firewall and security group rules.
  • IAM roles / service accounts used by BYOK8s environments and clusters.
  • Object storage buckets for data storage and log storage.
  • Network connectivity resources (NLBs and VPC Endpoint Services on AWS).
  • PostgreSQL databases serving as metadata stores.
  • Terraform state backend.
  • Encryption keys (KMS on AWS).

Features not available in BYOK8s

BYOK8s uses AWS PrivateLink for control plane connectivity (customer provisions the VPC Endpoint Services, control plane creates the VPC Endpoints). The features below refer to RisingWave-managed capabilities that are available in BYOC but not in BYOK8s.