BYOC vs BYOK8s
Architecture overview
In a BYOK8s environment, RisingWave deploys and manages the following components in your Kubernetes cluster:- CloudAgent: Handles operations sent by the RisingWave control plane (cluster provisioning, scaling, upgrades).
- RWProxy: Routes PostgreSQL protocol traffic from the control plane and user clients to the appropriate RisingWave instances.
- RisingWave Operator: Manages RisingWave Custom Resource lifecycle in Kubernetes.
- Self-hosted telemetry stack: VictoriaMetrics (metrics), Loki (logs), and Grafana Alloy (collection) — deployed automatically inside the BYOK8s cluster. No external observability services are required.
Prerequisites
Before setting up a BYOK8s environment, you must provision the following resources in your cloud account.1. Kubernetes cluster
- AWS
- GCP
- Provider: Amazon EKS
- Version: Kubernetes 1.32 or higher
- Region: Must be in the same AWS region as the RisingWave Cloud control plane
- Cluster mode: Standard EKS or EKS Auto Mode
aws.eks_auto_mode setting is stored when you run rwc byok create and cannot be changed with rwc byok update. RisingWave does not support an in-place migration between standard EKS and EKS Auto Mode; create a new BYOK8s environment to change modes.2. Cluster dependencies
- AWS
- GCP
Install cert-manager v1.19.2 or later in both cluster modes. The remaining dependencies differ by mode:
In EKS Auto Mode, RisingWave creates native
eks.amazonaws.com/v1 TargetGroupBinding resources. In a standard EKS cluster, RisingWave creates elbv2.k8s.aws/v1beta1 TargetGroupBinding resources that the self-managed AWS Load Balancer Controller reconciles.3. Object storage
Provision two separate storage buckets in the same region as your Kubernetes cluster.We recommend separate buckets so that data and logs can have independent lifecycle policies, access controls, and retention. The data store typically contains sensitive cluster state, while the log store contains observability data with different access and retention requirements.
- AWS
- GCP
Provide the S3 bucket ARNs for both buckets.
4. Encryption at rest
- AWS
- GCP
Provide a KMS key ARN for EBS encryption. This is used by VictoriaMetrics and Loki persistent volumes, RisingWave compute cache storage, and encrypted EKS worker-node storage.For standard EKS, grant the IAM principal used by the Amazon EBS CSI controller the required KMS permissions.For EKS Auto Mode, the key policy must grant:
- Allow the EKS cluster IAM role and the
AWSServiceRoleForAutoScalingservice-linked role to encrypt, decrypt, and re-encrypt data; generate data keys; describe the key; and create grants for AWS resources. - Allow the EKS cluster IAM role to create, list, and revoke grants for AWS resources.
- Allow principals in the same AWS account that are authorized to use EBS to use the key through EC2. Restrict this statement with
kms:CallerAccountandkms:ViaServiceset toec2.<region>.amazonaws.com.
NodeClass does not need to be an explicit key-policy principal when the account-scoped EBS condition is present. See the reference Terraform example for a working policy.5. Terraform state backend
- AWS
- GCP
Provision an S3 bucket and a DynamoDB table for storing BYOK8s environment Terraform state and state locking. CloudAgent uses this backend to manage Terraform resources inside your Kubernetes cluster, and the IAM policy in the next step references both.
6. Identity and access management
- AWS
- GCP
You must create the following IAM roles:A. Setup IAM (user context)This role is used by the person or CI/CD pipeline running the BYOK8s setup commands. It is not used by any in-cluster workloads.Where
- Must have
eks:DescribeClusterpermission on the target EKS cluster. - Must have an access entry in the EKS cluster with the
AmazonEKSClusterAdminPolicyaccess policy attached.
$TFSTATE_BUCKET and $TFLOCK_TABLE correspond to the S3 bucket and DynamoDB table provisioned in Terraform state backend.C. Loki IAM (service context)This role is assumed by Loki via IRSA for reading and writing logs to S3. Minimum permissions:7. Network connectivity
- AWS
- GCP
Provision two internal Network Load Balancers with VPC Endpoint Services for PrivateLink connectivity from the RisingWave control plane.CloudAgent NLB — one target group per port:
RWProxy NLB — one target group per port. The webhook listener port differs from its target port (the control plane reaches it on
443 over TLS; the RWProxy pod listens on 4580); pgwire and metrics use the same port on both sides.For each NLB, create a VPC Endpoint Service and allow the RisingWave control plane AWS account (
600598779918) as an allowed principal.EKS Auto Mode load-balancing requirements
EKS Auto Mode reconciles the pre-created target groups through its nativeTargetGroupBinding API. In addition to the shared NLB requirements above:- Add the tag
eks:eks-cluster-name=<cluster-name>to all five target groups. - Attach a target-traffic security group to both NLBs, and allow outbound TCP traffic from it to ports
40001,40090,4566,4580, and9099. - Allow inbound TCP traffic from the target-traffic security group to the EKS cluster primary security group on the same five ports.
- If clients connect directly to the RWProxy NLB, attach a separate client-access security group only to that NLB and allow inbound TCP traffic on port
4566from the approved client CIDRs or security groups. - Ensure every Auto Mode
NodeClassselects the EKS cluster primary security group. - Set the NLB attribute
enforce_security_group_inbound_rules_on_private_link_traffictoofffor the PrivateLink traffic used by this setup.
rwproxy_additional_client_cidrs for clients that arrive through peering, Transit Gateway, or another routed network.Standard EKS Terraform example for NLBs and VPC Endpoint Services
Standard EKS Terraform example for NLBs and VPC Endpoint Services
Create a BYOK8s environment
BYOK8s uses therwc CLI to provision and manage environments. Before continuing, install and authenticate the CLI by following Install the RisingWave Cloud CLI.
Step 1: Prepare the configuration file
Create a YAML configuration file with your infrastructure details:- AWS
- GCP
For a standard EKS cluster, leave
aws.eks_auto_mode set to false. For EKS Auto Mode, set it to true. The value defaults to false when omitted, must match the existing cluster mode when you register the environment, and cannot be changed later with rwc byok update.Optional: pod scheduling and custom settings
Optional: pod scheduling and custom settings
You can optionally configure pod scheduling for different workload types and add custom tags:See Node pool resource requirements for sizing guidance on each workload category.
Step 2: Register the environment
Step 3: Deploy resources
Ready.
Create a RisingWave cluster
Creating a RisingWave cluster in a BYOK8s environment uses a two-phase provisioning flow. This is because the IRSA trust policy for the cluster’s IAM role requires the Kubernetes namespace and service account name, which are only allocated after the cluster is initially created.Cluster creation in a BYOK8s environment is currently available only via the
rwc CLI. RisingWave Cloud portal support is planned but not yet implemented. Once the cluster is provisioned, you can manage it (including rescaling components and adjusting replicas) from the RisingWave Cloud portal as usual.Phase 1: Create the cluster
Create the cluster via therwc CLI. A BYOK8s cluster must use --tier BYOK together with per-component sizing flags, and the --env flag must point to the BYOK8s environment registered with rwc byok create. The metastore is not configured at this step — the customer-managed PostgreSQL connection details are supplied in Phase 2 via rwc cluster byok-config.
--name— name of the new RisingWave cluster.--tier BYOK— required tier for clusters running in a BYOK8s environment.--env— name of the BYOK8s environment registered withrwc byok create.--compute/--compactor/--frontend/--meta— component-type IDs that match instance shapes available in your Kubernetes cluster. Component IDs follow the patternp-<X>c<Y>g, whereXis the number of CPU cores andYis the amount of memory in GiB (for example,p-4c16gmeans 4 CPU and 16 GiB memory). The values above are a small example sizing (1 replica each atp-4c16gfor compute andp-2c8gfor the rest); replace them to match your workload. Runrwc cluster create --helpfor the full flag list.--*-replica— replica count for each component.
AwaitingConfig status. The success message from rwc cluster create includes the cluster’s UUID and the exact rwc cluster byok-config command to run next; you can also list existing clusters and their UUIDs with rwc cluster list.
At this point, the control plane has allocated the Kubernetes namespace and service account name. Retrieve them with:
Resource Namespace and Service Account fields in the output. You will need these values for the IAM trust policy in the next step.
Phase 2: Provision IAM and metastore, then configure
- AWS
- GCP
-
Create an IAM role for the RisingWave cluster with an IRSA trust policy. Use the
Resource NamespaceandService Accountvalues from therwc cluster describeoutput in Phase 1:Grant S3 access to the cluster’s data and backup prefixes in the data store bucket. RisingWave stores cluster state underdata-$RESOURCE_NAMESPACE/and meta backups underdata-$RESOURCE_NAMESPACE-backup/, so both prefixes must be included: - Provision a PostgreSQL database as the metadata store. We recommend using Amazon RDS. Ensure connectivity from the EKS cluster pods to the database.
-
Submit the configuration via CLI:
Config file format:
Creating status and triggers provisioning. The metastore password is encrypted at rest.Connect to a RisingWave cluster
There are two ways to connect to a RisingWave cluster running in a BYOK8s environment.Option 1: RisingWave Cloud Console (recommended)
Use the in-portal SQL console — no network setup is required. See Console overview for details.Option 2: Direct connection via RWProxy NLB
For programmatic access (e.g., from your own applications, BI tools, orpsql), connect to the RWProxy NLB you provisioned in the prerequisites.
The RWProxy NLB is internal-only. Clients must be inside the same VPC as the BYOK8s environment, or reach the NLB via VPC peering / Transit Gateway.
For EKS Auto Mode, attach a client-access security group only to the RWProxy NLB before using direct connections. Allow inbound TCP traffic on port
4566 from the approved client CIDRs or security groups. The reference Terraform example allows its VPC CIDR by default and accepts additional routed CIDRs through rwproxy_additional_client_cidrs. Disabling NLB security-group enforcement for PrivateLink traffic does not allow direct VPC client traffic.options field, the host (SNI), or the username. See Connection errors → Tenant identifier methods for all three methods. Example using the options field:
%3D is the URL-encoded form of =. If your client does not require URL encoding, use --tenant=<cluster-namespace> directly.
Where:
<rwproxy-nlb-dns>— DNS name of the RWProxy NLB (the load balancer behindrwproxy_target_group_arn)4566— RWProxy PostgreSQL port<database>— database name (default:dev)<cluster-namespace>— theResource Namespacefromrwc cluster describe<username>/<password>— credentials created viarwc cluster dbuser create
Node pool resource requirements
If you use dedicated node pools for different workload types (via thecustomized_settings.scheduling configuration), the following minimum resources are required:
The same workload categories and minimum sizing apply to both standard EKS and EKS Auto Mode:
DaemonSet scheduling (Grafana Alloy) is automatically derived from the union of all workload tolerations — no separate configuration is needed.
Manage a BYOK8s environment
List environments
View environment details
Update an environment
To update a BYOK8s environment (e.g., to apply a new version or change custom settings):Delete a BYOK8s environment
- Delete all RisingWave clusters running in the environment.
-
Terminate the environment (control plane side):
-
Delete the Kubernetes resources and control plane record:
- Clean up the cloud resources you provisioned (NLBs, VPC Endpoint Services, IAM roles, storage buckets, etc.).
Shared responsibility
BYOK8s is a shared responsibility model. The table below outlines what each party is responsible for.RisingWave responsibilities
- All cluster and project-level operations available on the RisingWave Cloud portal.
- 24/7 monitoring backed by the RisingWave support and on-call teams.
- Version and configuration management of all deployed workloads (CloudAgent, RWProxy, Operator, telemetry stack).
Customer responsibilities
- Kubernetes cluster and worker nodes (including upgrades to the minimum supported version).
- Firewall and security group rules.
- IAM roles / service accounts used by BYOK8s environments and clusters.
- Object storage buckets for data storage and log storage.
- Network connectivity resources (NLBs and VPC Endpoint Services on AWS).
- PostgreSQL databases serving as metadata stores.
- Terraform state backend.
- Encryption keys (KMS on AWS).
Features not available in BYOK8s
BYOK8s uses AWS PrivateLink for control plane connectivity (customer provisions the VPC Endpoint Services, control plane creates the VPC Endpoints). The features below refer to RisingWave-managed capabilities that are available in BYOC but not in BYOK8s.