Object storage configuration

Configure the object storage backend (such as AWS S3, GCS, Azure Blob Storage, or Azure Data Lake Storage Gen2) for your Iceberg warehouse.

This guide provides the configuration parameters for connecting RisingWave to the object storage backend for your Apache Iceberg data. These parameters are used in the WITH clause when creating an Iceberg source, sink, or connection.

S3-compatible storage

These parameters configure the connection to an S3-compatible storage system, such as AWS S3 or MinIO, where your Iceberg data files are stored.

Parameter	Description
warehouse.path	Required. The base path to your Iceberg warehouse. Example: 's3://my-bucket/iceberg-warehouse'
s3.region	Required. The AWS region where the bucket is hosted.
s3.access.key	Required when enable_config_load = false. The AWS access key ID.
s3.secret.key	Required when enable_config_load = false. The AWS secret access key.
s3.iam_role_arn	Optional. The IAM role ARN to assume for S3 access via STS. Can be used with either enable_config_load = false or true.
enable_config_load	Optional. If set to true, load AWS config/credentials from the environment (for example, to assume s3.iam_role_arn). When enable_config_load = true, do not provide s3.access.key/s3.secret.key.
s3.endpoint	Optional. The endpoint for S3-compatible services like MinIO. For AWS S3, this is typically not needed.
s3.path.style.access	Optional. Set to true to use path-style access (e.g., for MinIO). Defaults to false for virtual-hosted–style access.

In RisingWave Cloud, you have two options:

• Provide s3.access.key and s3.secret.key, or
• Use IAM role delegation: grant S3 permissions to your own IAM role, allow the RisingWave Cloud tenant role to assume it, then set s3.iam_role_arn = '<your_role_arn>' and enable_config_load = true. See Set up IAM role assume for a step-by-step guide, including how to retrieve the required IAM role ARN from the Cloud metadata page.

If you use a REST catalog with vended credentials (vended_credentials = true), you can omit S3 credentials here because the catalog server provides temporary credentials.

In self-hosted deployments, if you want to rely on the AWS SDK default credential chain (for example, environment variables or an EC2 instance profile), set enable_config_load = true and omit s3.access.key/s3.secret.key.

Google Cloud Storage (GCS)

These parameters configure the connection to Google Cloud Storage.

Support for GCS was added in RisingWave v2.3.0.

Parameter	Description
warehouse.path	Required. The GCS path to your warehouse. Example: 'gs://my-bucket/iceberg-warehouse'
gcs.credential	Optional. The Base64-encoded credential key from a GCS service account JSON file. If not provided, Application Default Credentials (ADC) will be used.

Example

CREATE SINK my_iceberg_sink FROM my_mv WITH (
    connector = 'iceberg',
    type = 'append-only',
    catalog.type = 'rest',
    catalog.uri = 'http://127.0.0.1:8181',
    warehouse.path = 'gs://my-bucket/warehouse',
    gcs.credential = '...'
);

Azure Blob Storage

These parameters configure the connection to Azure Blob Storage.

Support for Azure Blob Storage was added in RisingWave v2.4.0.

Parameter	Description
warehouse.path	Required. The Azure Blob Storage path. Example: 'azblob://container-name/warehouse'
azblob.account_name	Required. The Azure Storage account name.
azblob.account_key	Required. The Azure Storage account key.
azblob.endpoint_url	Optional. The endpoint URL for the Azure Blob service. Defaults to https://<account_name>.blob.core.windows.net/.

Example

CREATE SINK my_iceberg_sink FROM my_mv WITH (
    connector = 'iceberg',
    type = 'append-only',
    catalog.type = 'storage',
    warehouse.path = 'azblob://my-container/warehouse',
    azblob.account_name = 'myaccount',
    azblob.account_key = '...'
);

Azure Data Lake Storage Gen2 (ADLS)

These parameters configure the connection to Azure Data Lake Storage Gen2.

Support for ADLS was added in RisingWave v2.5.0. ADLS can only be used with REST catalog.

Parameter	Description
warehouse.path	Required. The warehouse path. This is typically just the name of the container or a relative path.
adlsgen2.account_name	Required. The Azure Storage account name.
adlsgen2.account_key	Required. The Azure Storage account key. Alternatively, you can use client credentials (client ID, client secret, tenant ID) configured in your REST catalog.
adlsgen2.endpoint	Optional. The endpoint URL for ADLS. Defaults to https://<account_name>.dfs.core.windows.net/.

Example

CREATE SINK my_iceberg_sink FROM my_mv WITH (
    connector = 'iceberg',
    type = 'upsert',
    primary_key = 'id',
    catalog.type = 'rest',
    catalog.uri = 'http://127.0.0.1:8181/catalog/',
    catalog.name = 'demo',
    warehouse.path = 'test',
    database.name = 'my_database',
    table.name = 'my_table',
    adlsgen2.account_name = 'myaccount',
    adlsgen2.account_key = '...'
);