Hive Metastore catalog

Connect RisingWave to an Apache Hive Metastore for Apache Iceberg.

You can use an Apache Hive Metastore as the catalog for your Iceberg tables. You can use this catalog with RisingWave when creating an Iceberg SOURCE, SINK, or CONNECTION (for internal Iceberg tables).

Hive Metastore catalog is currently in the technical preview stage.

For a complete list of all catalog-related parameters, see the main Catalog configuration page.

Example

Examples for using a Hive Metastore catalog with internal Iceberg tables, sinks, and sources.

Internal tables (recommended)

CREATE CONNECTION hive_conn WITH (
    type = 'iceberg',
    warehouse.path = 's3://my-bucket/warehouse/',
    s3.region = 'us-west-2',
    s3.access.key = '...',
    s3.secret.key = '...',
    enable_config_load = false,

    catalog.type = 'hive',
    catalog.uri = 'thrift://hive-metastore:9083'
);

SET iceberg_engine_connection = 'public.hive_conn';

-- `commit_checkpoint_interval` controls Iceberg commit frequency. Default: about every 60 seconds; set to 1 for faster commits and visibility.
CREATE TABLE my_internal_iceberg_table (
  id INT PRIMARY KEY,
  name VARCHAR
)
WITH (commit_checkpoint_interval = 1)
ENGINE = iceberg;

Write (CREATE SINK)

CREATE SINK my_hive_sink FROM my_mv WITH (
    connector = 'iceberg',
    type = 'upsert',
    primary_key = 'id',
    catalog.type = 'hive',
    catalog.uri = 'thrift://hive-metastore:9083',
    warehouse.path = 's3://my-bucket/warehouse/',
    s3.region = 'us-west-2',
    s3.access.key = '...',
    s3.secret.key = '...',
    database.name = 'my_db',
    create_table_if_not_exists = 'true',
    table.name = 'my_table'
);

Read (CREATE SOURCE)

CREATE SOURCE my_hive_source
WITH (
    connector = 'iceberg',
    catalog.type = 'hive',
    catalog.uri = 'thrift://hive-metastore:9083',
    warehouse.path = 's3://my-bucket/warehouse/',
    s3.region = 'us-west-2',
    s3.access.key = '...',
    s3.secret.key = '...',
    database.name = 'my_db',
    table.name = 'my_table'
);