Amazon S3 Tables catalog

Connect RisingWave to Amazon S3 Tables as a REST-based catalog for Apache Iceberg.

Amazon S3 Tables is an AWS-native Iceberg catalog service that uses the REST protocol with AWS SigV4 authentication. You can use it with RisingWave when creating an Iceberg SOURCE, SINK, or CONNECTION (for internal Iceberg tables).

New to S3 Tables and need to create a table bucket / namespace / table first? See Create Amazon S3 Tables with AWS CLI.

Required REST parameters for S3 Tables:

Parameter	Description	Value for S3 Tables
catalog.uri	The S3 Tables REST endpoint for your region.	https://s3tables.<your-region>.amazonaws.com/iceberg
catalog.rest.signing_region	The AWS region for signing requests.	e.g., us-east-1
catalog.rest.sigv4_enabled	Enables SigV4 signing. Must be true.	true
catalog.rest.signing_name	The service name for signing requests.	s3tables

For a complete list of all catalog-related parameters, see the main Catalog configuration page.

Internal tables (recommended)

Use S3 Tables as the catalog for RisingWave-managed (internal) Iceberg tables by creating an Iceberg CONNECTION, then creating tables with ENGINE = iceberg.

CREATE CONNECTION s3_tables_conn WITH (
    type = 'iceberg',
    warehouse.path = 'arn:aws:s3tables:<your-region>:<account-id>:bucket/<bucket-name>',
    s3.region = '<your-region>',
    s3.access.key = '...',
    s3.secret.key = '...',
    enable_config_load = false,

    catalog.type = 'rest',
    catalog.uri = 'https://s3tables.<your-region>.amazonaws.com/iceberg',
    catalog.rest.signing_region = '<your-region>',
    catalog.rest.sigv4_enabled = true,
    catalog.rest.signing_name = 's3tables'
);

SET iceberg_engine_connection = 'public.s3_tables_conn';

-- `commit_checkpoint_interval` controls Iceberg commit frequency. Default: about every 60 seconds; set to 1 for faster commits and visibility.
CREATE TABLE my_internal_iceberg_table (
  id INT PRIMARY KEY,
  name VARCHAR,
  value INT
)
WITH (commit_checkpoint_interval = 1)
ENGINE = iceberg;

Write (CREATE SINK)

Deliver data from RisingWave to an S3 Tables-managed Iceberg table using CREATE SINK.

CREATE SINK my_s3_tables_sink FROM my_mv
WITH (
    connector = 'iceberg',
    type = 'upsert',
    primary_key = 'id',
    warehouse.path = 'arn:aws:s3tables:<your-region>:<account-id>:bucket/<bucket-name>',
    s3.access.key = '...',
    s3.secret.key = '...',
    s3.region = '<your-region>',
    catalog.type = 'rest',
    catalog.uri = 'https://s3tables.<your-region>.amazonaws.com/iceberg',
    catalog.rest.signing_region = '<your-region>',
    catalog.rest.sigv4_enabled = true,
    catalog.rest.signing_name = 's3tables',
    database.name = '<your-database-name>',
    create_table_if_not_exists = 'true',
    table.name = '<your-table-name>'
);

Read (CREATE SOURCE)

Ingest data from an S3 Tables-managed Iceberg table into RisingWave using CREATE SOURCE.

CREATE SOURCE my_s3_tables_source
WITH (
    connector = 'iceberg',
    warehouse.path = 'arn:aws:s3tables:<your-region>:<account-id>:bucket/<bucket-name>',
    s3.access.key = '...',
    s3.secret.key = '...',
    s3.region = '<your-region>',
    catalog.type = 'rest',
    catalog.uri = 'https://s3tables.<your-region>.amazonaws.com/iceberg',
    catalog.rest.signing_region = '<your-region>',
    catalog.rest.sigv4_enabled = true,
    catalog.rest.signing_name = 's3tables',
    database.name = '<your-database-name>',
    table.name = '<your-table-name>'
);